diff --git a/AGENTS.md b/AGENTS.md index d87b9819..5e249054 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -96,6 +96,16 @@ Written acceptance procedures: Proves Desktop traffic reaches `ai_gateway_messages` by both the live gateway route and the `~/.codex/sessions` backfill route, and is attributable via `entrypoint`. See `docs/ACCEPTANCE.md`. +- `openclaw_capture`: opt-in/manual, needs OpenClaw with both `anthropic` and + `openai` credentials. Proves both capture lanes (live gateway and the + scheduled transcript sweep) and that a turn both lanes observe settles to + one row. See `docs/ACCEPTANCE.md`. +- `claude_otel_shape_check`: opt-in/manual, needs a real Claude Code 2.1.214 + or newer. The release gate against upstream drift on the OTEL attach path: + proves the installed Claude Code still honors the managed `env` block and + still emits the event names, attributes, and raw body fields the telemetry + listener reads, then checks the rows and the `hyp status` capture-health + line agree. See `docs/ACCEPTANCE.md`. Good acceptance smoke candidates (no written procedure yet): @@ -171,6 +181,7 @@ src/ cli/ # dispatch, walkthrough, core_commands config/ # v2 schema, validator daemon/ # platform installers (launchd / systemd) + lifecycle + otlp/ # shared OTLP http/json listener machinery plugin_install/ # resolver, fetch, lock, update_check sinks/ # cron driver + encoder utility hypaware-core/ @@ -210,6 +221,7 @@ hyp smoke walkthrough_picker_to_first_query hyp smoke client_attach_idempotent hyp smoke gateway_claude_capture hyp smoke gateway_codex_capture +hyp smoke claude_telemetry_capture hyp smoke hypignore_capture_drop hyp smoke local_only_export_withhold hyp smoke source_optout_export_withhold @@ -236,6 +248,18 @@ If the release touched a client adapter, run the matching procedure in [`docs/ACCEPTANCE.md`](docs/ACCEPTANCE.md) and record the result in the release notes. +If the release touched the **claude** adapter (`@hypaware/claude`, the +telemetry listener, the body spool, or the attach settings writer), the +matching procedure is +[`claude_otel_shape_check`](docs/ACCEPTANCE.md#claude_otel_shape_check). It is +not optional for those releases and it is not substitutable by the hermetic +smokes: `claude_telemetry_capture` POSTs a fixture we wrote, so it agrees with +itself no matter what upstream did. Only a real Claude Code can tell you it +renamed an event, dropped a flag, or changed the raw body format, and the +failure mode is silent (null columns, not an error). Record the observed +`claude --version` and the full event-name list in the release notes so the +next release has a baseline to diff against. + ## LLP conventions diff --git a/CONTEXT.md b/CONTEXT.md index 03478090..561a7b6b 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -15,10 +15,9 @@ offers `claude`, `codex`, and `otel`, the two raw proxy rows being **hidden** - **Client source**: a known tool HypAware configures for you. `claude` and `codex` are the client sources. Picking one adds its gateway upstream *and* - its adapter plugin (`@hypaware/claude` / `@hypaware/codex`), which attaches - the tool (rewrites its base URL), installs hooks/skills, and can backfill - its local history. Client sources are the only sources that can be - [[autodetect]]ed. + its adapter plugin (`@hypaware/claude` / `@hypaware/codex`), which + [[attach]]es the tool, installs hooks/skills, and can backfill its local + history. Client sources are the only sources that can be [[autodetect]]ed. - **Raw proxy source**: `raw-anthropic` / `raw-openai`. Picking one opens the gateway with that provider upstream but configures no client; the user points their own SDK app or script at the local gateway by hand. Serves the @@ -31,6 +30,38 @@ offers `claude`, `codex`, and `otel`, the two raw proxy rows being **hidden** OpenTelemetry signals. Like a raw proxy source, it is manual and not autodetectable. +An `otel` picker source is not the same thing as "a source that speaks OTLP". +`@hypaware/claude` runs its own OTLP listener to receive Claude Code's +telemetry ([[attach]] mode `otel`, LLP 0257), on its own port, with its own +payload rules and its own datasets. That listener is claude-owned: a machine +attached that way still has `claude` as a **client source** here, autodetected +and configured for the user, and nothing about it turns on the `otel` source. +Picking `otel` is what a user does for *their own* app's telemetry. + +### Attach + +Writing a reversible block into a **client source**'s own configuration so that +what the tool does reaches HypAware, and being able to take it back out. +`hyp attach ` writes it, `hyp detach ` removes exactly those +keys and restores anything they displaced, and the undo record (the `_hypaware` +marker) lives in the file that was edited. + +Attach is not one mechanism. Each client adapter picks a **mode**, and +`hyp status` names it (`claude [configured, attached (otel)]`): + +- **`base_url`**: point the tool's API base URL at the local gateway. `codex` + attaches this way. +- **`proxy`**: set `HTTPS_PROXY` and trust a machine-local CA, so the gateway + sees the tool's TLS traffic without its base URL being touched (LLP 0232). +- **`otel`**: turn on the tool's own OpenTelemetry export and point it at a + HypAware listener (LLP 0258). No base URL, no proxy, no CA: the tool still + talks straight to its provider and HypAware receives a copy of what it did. + `claude` attaches this way. + +The mode is worth naming because it decides what being attached costs: only +`proxy` installs CA trust, and only `base_url` and `proxy` put the HypAware +daemon on the request path. + ### Autodetect The first-run wizard inspecting the system for the presence of a **client diff --git a/README.md b/README.md index 6584b648..5a84f60f 100644 --- a/README.md +++ b/README.md @@ -276,16 +276,46 @@ client's own config file (for example `~/.claude/settings.json` for Claude, a `hypaware` provider entry in `~/.codex/config.toml` for Codex); unrelated keys in every file are preserved. -### Proxy mode (keeps Claude Code's Remote Control working) - -By default `hyp attach claude` points `ANTHROPIC_BASE_URL` at the local -gateway. Claude Code disables **Remote Control** whenever that variable -points anywhere other than `api.anthropic.com`, so an attached machine -loses it. - -Proxy mode avoids that by leaving the base URL alone and routing Claude -Code through the gateway as an HTTPS proxy instead. Turn it on in the -`ai-gateway` section of `~/.hyp/hypaware-config.json` and restart the +### Claude Code attaches by telemetry, not by proxy + +`hyp attach claude` writes one reversible `env` block into +`~/.claude/settings.json` that turns on Claude Code's own OpenTelemetry +export and points it at a loopback listener the daemon runs. It leaves +`ANTHROPIC_BASE_URL` alone, sets no proxy, and installs no certificate +authority, so Claude Code still talks straight to `api.anthropic.com`, +**Remote Control keeps working**, and a daemon that is down or wedged costs +you capture rather than your session. Nothing has to be quit and reopened: +Claude Code reads the `env` block at launch, on every launch path. + +Two things ride along with the conversation rows: + +- **Raw request and response bodies** land in `~/.hyp/spool/claude-bodies` + (owner-only) until the listener projects them and deletes them. They carry + what the events do not: the system prompt, the tool list, and untruncated + tool arguments. The directory is capped (512 MB by default, oldest evicted + first), and both `hyp purge` and `hyp detach claude` empty it. +- **Behavioral signals the wire never showed** land in their own + `claude_telemetry_events` table: tool accept and reject decisions, + permission mode changes, per-request cost, hook and MCP health. + +Claude Code **2.1.193 or newer** is required (2.1.214 for the full +tool-decision detail). Below the floor, attach refuses the switch, leaves any +existing attach exactly as it is, and prints `claude update`, rather than +silently capturing less. + +`hyp detach claude` removes exactly those keys, restores anything they +displaced, and sweeps the spool. + +If this machine was attached by proxy before, `hyp attach claude` is also the +migration: it releases the proxy keys, unwinds the launchd environment, and +tells you how to end the CA trust that it will not end for you +(`hyp detach claude --purge`). + +### Proxy mode (TLS interception for the clients that still proxy) + +Claude Code no longer uses this path. It remains how the gateway captures a +client that cannot simply be pointed at a different base URL. Turn it on in +the `ai-gateway` section of `~/.hyp/hypaware-config.json` and restart the daemon: ```json @@ -294,37 +324,35 @@ daemon: ```sh hyp daemon restart -hyp attach claude ``` -On the next attach, HypAware sets `HTTPS_PROXY` and `NODE_EXTRA_CA_CERTS` -instead of the base URL. What this changes: +Such a client is then pointed at the gateway with `HTTPS_PROXY` and +`NODE_EXTRA_CA_CERTS` rather than a base URL. What that changes: - **A machine-local certificate authority is generated** under `~/.hyp/hypaware/tls`, readable only by you, and name-constrained so it cannot vouch for any host outside the provider set HypAware intercepts. - On macOS, attach also adds it to your **login keychain** as a user-domain - trusted root, because Claude Code's Remote Control transport trusts only - the keychain: macOS raises its own password dialog, and declining it - leaves capture working with Remote Control's inbound channel off. No admin - rights are needed and the machine-wide system keychain is not touched. On - other platforms trust stays file-scoped to Claude Code's own settings. - `hyp status` shows the fingerprint and whether the keychain still trusts - it. `hyp detach claude` keeps the CA and the trust, so re-attaching does - not ask again; `hyp detach claude --purge` and `hyp daemon uninstall` - remove both. -- **Only `api.anthropic.com` is decrypted**, because that is the only host - a registered upstream names. Every other host Claude Code talks to is - tunnelled through without being decrypted. -- **What gets recorded does not change.** Only `/v1/messages` is recorded, - exactly as before; the other paths Claude Code calls on that host are - passed through without being stored. + On macOS it can also be added to your **login keychain** as a user-domain + trusted root, for a client whose transport trusts only the keychain: macOS + raises its own password dialog, and declining it leaves capture working + with that inbound channel off. No admin rights are needed and the + machine-wide system keychain is not touched. On other platforms trust + stays file-scoped to the client's own settings. `hyp status` shows the + fingerprint and whether the keychain still trusts it. + `hyp detach ` keeps the CA and the trust, so re-attaching does not + ask again; `hyp detach --purge` and `hyp daemon uninstall` remove + both. +- **Only the hosts a registered upstream names are decrypted.** Every other + host the client talks to is tunnelled through without being decrypted. +- **What gets recorded does not change.** Only the recorded API paths are + stored; the other paths a client calls on the same host are passed through + without being stored. Two things to know before turning it on: -- If the daemon is not running, Claude Code's HTTPS all fails, not just its - model calls. Attach refuses to write the settings unless proxy mode is - actually running, and `hyp detach claude` is the escape hatch. +- If the daemon is not running, a proxied client's HTTPS all fails, not just + its model calls. Attach refuses to write the settings unless proxy mode is + actually running, and `hyp detach ` is the escape hatch. - If you already use a corporate proxy, set `upstream_proxy` to it so traffic still chains through it. Attach warns and backs up your existing `HTTPS_PROXY` (restored on detach) rather than silently replacing it: diff --git a/docs/ACCEPTANCE.md b/docs/ACCEPTANCE.md index cbacae8f..6b199ba5 100644 --- a/docs/ACCEPTANCE.md +++ b/docs/ACCEPTANCE.md @@ -515,6 +515,308 @@ procedure checks, R11 in particular), [LLP 0172](../llp/0172-openclaw-two-lane-c --- +## `claude_otel_shape_check` + +**What it proves:** that the **installed** Claude Code still emits the +telemetry HypAware's `otel` attach depends on: the nine-key `env` block is +honored, the expected event names arrive with the attributes the listener +reads, the raw body files still carry the fields the projector fills its +column gaps from, and the whole path lands `ai_gateway_messages` and +`claude_telemetry_events` rows with nothing null that should not be. + +This is the release-gate half of LLP 0262's flag-stability duty (open +question 5). The other half runs in production: the `hyp status` capture-health +line. Neither can be replaced by a hermetic smoke, because a smoke POSTs a +fixture we wrote and therefore agrees with itself forever. Only a real Claude +Code can tell you it renamed an event or dropped a flag. + +**What it does not prove:** anything about the gateway proxy path (still the +capture route for `codex`, `claude-desktop`, `openclaw`, `hermes`, and raw SDK +traffic), anything about fleet managed-settings delivery, anything about +central forwarding, or anything on a machine other than the one you ran it on. + +**Requires:** + +- A real Claude Code install, **2.1.214 or newer**. Attach's own floor is + 2.1.193 (the event set) and it refuses the mode switch below it + ([LLP 0258#version-floor](../llp/0258-attach-injects-telemetry-via-settings-env.decision.md#version-floor)), + so there is nothing to shape-check there. This procedure asks for the higher + number because step 8 asserts the tool-decision `source`, which arrives at + 2.1.214: between the two versions attach succeeds and that one field reads + null, which is correct behavior and would read here as a false failure. +- HypAware installed from the package under test, `@hypaware/claude` enabled, + daemon running, and `jq` on `PATH`. +- A scratch git repository to hold the two conversations in. Do not run this in + a directory covered by `.hypignore` or the machine-local list: the usage + policy drops those sessions at ingest by design, and every row assertion + below would then fail for the right reason at the wrong time. +- Willingness to have two short real conversations recorded on this machine. + +**Related:** +[LLP 0262](../llp/0262-otel-attach-replaces-proxy.rfc.md) (the design record and +open question 5), +[LLP 0257](../llp/0257-claude-telemetry-listener-source.spec.md) (S21, the +two-layer drift detection this discharges), +[LLP 0258](../llp/0258-attach-injects-telemetry-via-settings-env.decision.md) +(the env keys step 1 asserts), +[LLP 0252](../llp/0252-events-carry-content-bodies-fill-the-gaps.decision.md) +(which fields come from events and which from bodies), +[LLP 0253](../llp/0253-body-spool-is-capped-and-swept.decision.md) (the spool), +[LLP 0255](../llp/0255-claude-telemetry-events-dataset.decision.md) (the +`claude_telemetry_events` row shape). + +### Steps + +1. Attach, and confirm the env block on disk is exactly the managed key set: + + ```sh + SETTINGS="${CLAUDE_HOME:-$HOME/.claude}/settings.json" + claude --version + hyp attach claude + jq '.env, ._hypaware' "$SETTINGS" + hyp status + ``` + + Pass condition: `claude --version` is 2.1.214 or newer; `env` carries all + nine managed keys (`CLAUDE_CODE_ENABLE_TELEMETRY`, `OTEL_LOGS_EXPORTER`, + `OTEL_METRICS_EXPORTER`, `OTEL_EXPORTER_OTLP_PROTOCOL`, + `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_LOG_USER_PROMPTS`, + `OTEL_LOG_ASSISTANT_RESPONSES`, `OTEL_LOG_TOOL_DETAILS`, + `OTEL_LOG_RAW_API_BODIES`) and **no** `ANTHROPIC_BASE_URL`, `HTTPS_PROXY`, + or `NODE_EXTRA_CA_CERTS`; the `_hypaware` marker records `mode: "otel"` and + the spool directory; `hyp status` shows + `claude [configured, attached (otel)]` and a running daemon. + +2. Pin the window, so every later query measures only this run: + + ```sh + SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ) + SINCE_SQL=${SINCE%Z} + SPOOL="${HYP_HOME:-$HOME/.hyp}/spool/claude-bodies" + ls -ld "$SPOOL" + ``` + + Pass condition: the spool directory exists and reads `drwx------` + ([LLP 0253#spool-location](../llp/0253-body-spool-is-capped-and-swept.decision.md#spool-location)). + `$SINCE` is the ISO instant; `$SINCE_SQL` is the same instant without the + zone suffix, which is what compares cleanly against a TIMESTAMP column. + +3. Take a raw body sample with the daemon **stopped**. Stopping it is what + makes this step deterministic: nothing consumes the spool, so the files sit + still long enough to read, which the live path never allows (a projected + body is deleted immediately). + + ```sh + hyp daemon stop + ``` + + Now, in the scratch repo, hold one short Claude Code conversation in a + **fresh** session (the settings `env` applies at launch, so a session that + was already open is not attached). Ask it to read one file, so the request + carries a tool definition and the response a tool-use block. Then: + + ```sh + mkdir -p /tmp/hyp-shape-check && cp "$SPOOL"/* /tmp/hyp-shape-check/ + for f in /tmp/hyp-shape-check/*; do echo "== $f"; jq -r 'keys | join(",")' "$f"; done + ``` + + Pass condition: at least two files, and their top-level key lists identify a + request body and a response body. If the directory is empty, stop here: + `OTEL_LOG_RAW_API_BODIES` is no longer writing files, which is the single + biggest drift this procedure exists to catch. + +4. Assert the body shape. These are exactly the fields the projector reads a + body **for** + ([LLP 0252#bodies-for-gaps](../llp/0252-events-carry-content-bodies-fill-the-gaps.decision.md#bodies-for-gaps)): + everything else in the file the events already delivered. + + ```sh + REQ=$(grep -l '"messages"' /tmp/hyp-shape-check/* | head -1) + RES=$(grep -l '"stop_reason"' /tmp/hyp-shape-check/* | head -1) + jq '{model, system: (.system|type), messages: (.messages|type), + tools: (.tools|type), tool0: (.tools[0]|keys?)}' "$REQ" + jq '{id, role, model, stop_reason, + content: [.content[].type], usage: (.usage|keys)}' "$RES" + ``` + + Pass condition: the request body is a JSON object carrying `model`, a + `system` that is a string or an array of text blocks, a `messages` array, + and a `tools` array whose entries carry `name` and `input_schema`. The + response body carries `id`, `role`, `model`, `stop_reason`, a `content` + array of typed blocks, and a `usage` object. A missing field here is silent + column loss downstream (`system_text`, `tools`, untruncated `tool_args`), + not a crash, which is why it is asserted on the file rather than inferred + from a null column. + + Then clear the sample. Those bodies are orphans: their events were lost + with the daemon down, so nothing will ever project them and they would sit + in the spool until the byte cap evicted them. + + ```sh + rm -f "$SPOOL"/* + hyp daemon start + ``` + +5. Hold the real conversation, daemon up, in a **fresh** session in the same + scratch repo. Drive three things on purpose, because each one is a separate + event this procedure asserts: + + - let it run one tool call to completion (`tool_result`), + - **reject** one tool call when it asks (`tool_decision` with + `decision = reject`), + - change permission mode once, e.g. accept-edits (`permission_mode_changed`). + + Then wait out one export interval and confirm the spool drained. Claude + Code batches its exports, so an immediate check reads "not yet" as + "broken": + + ```sh + sleep 60 + ls -1 "$SPOOL" | wc -l + ``` + + Pass condition: `0`, or only the files of a turn still in flight. Bodies are + projected and then deleted + ([LLP 0252#project-then-delete](../llp/0252-events-carry-content-bodies-fill-the-gaps.decision.md#project-then-delete)), + so a spool that keeps growing means the listener is not consuming what + Claude Code writes. + +6. Assert the message rows, and that the body join actually filled its columns: + + ```sh + hyp query sql " + select role, + count(*) n, + max(message_created_at) last_seen, + sum(case when system_text is not null then 1 else 0 end) with_system, + sum(case when tools is not null then 1 else 0 end) with_tools, + sum(case when cwd is not null then 1 else 0 end) with_cwd, + sum(case when git_branch is not null then 1 else 0 end) with_branch, + sum(case when client_version is not null then 1 else 0 end) with_version + from ai_gateway_messages + where conversation_source = 'claude_code' + and message_created_at >= '$SINCE_SQL' + group by 1 + order by 1" + ``` + + Pass condition: rows for both `user` and `assistant`; `with_system` and + `with_tools` above zero (that is the body join, step 4's fields arriving in + columns); `with_cwd` and `with_branch` above zero (that is the SessionStart + hook, which is where cwd and git identity come from on this path, not the + events); `with_version` above zero (`app.version` off the events). + `with_cwd = 0` with everything else healthy means the hook is not installed + and the usage policy is running blind, which is a release blocker on its own. + +7. Assert the event names. This query is both the presence check and the drift + detector, because an event name the listener does not model is still + recorded rather than dropped + ([LLP 0257#failure-modes](../llp/0257-claude-telemetry-listener-source.spec.md#failure-modes)): + + ```sh + hyp query sql " + select event_name, count(*) n, max(event_timestamp) last_seen + from claude_telemetry_events + where event_timestamp >= '$SINCE_SQL' + group by 1 + order by 1" + ``` + + Pass condition: the list contains at least `api_request`, `tool_decision`, + `tool_result`, `permission_mode_changed`, and the metric rows + `claude_code.cost.usage`, `claude_code.lines_of_code.count`, and + `claude_code.active_time.total`. The metric rows ride the metrics exporter, + whose interval is longer than the logs one: if the `claude_code.*` names are + the only ones missing, wait another minute and re-run this query before + concluding anything. Write the **whole** list into the release + notes, not just the verdict: a name this document does not mention is an + upstream addition worth a follow-up, and a name that has stopped appearing + is upstream drift to file before the release ships. Note that `user_prompt`, + `assistant_response`, `api_request_body`, and `api_response_body` are + *expected to be absent here*: the first two are projected into + `ai_gateway_messages` and the last two are body pointers, so their absence + from this table is correct and their presence would be the bug. + +8. Assert the event attributes, which is where a flag going quiet shows up as a + null rather than an error: + + ```sh + hyp query sql --max-bytes 0 " + select event_name, tool_name, decision, source, cost_usd, attributes + from claude_telemetry_events + where event_timestamp >= '$SINCE_SQL' + and event_name in ('tool_decision', 'api_request', 'permission_mode_changed') + order by event_timestamp + limit 12" + ``` + + Pass condition: the `tool_decision` row for the call you rejected has + `decision = reject` and a non-null `source` (the 2.1.214 detail); the + `api_request` row has a non-null `cost_usd` and its `attributes` carry + `model`, `input_tokens`, `output_tokens`, and the cache-token pair; the + `permission_mode_changed` row's `attributes` carry `from_mode` and + `to_mode`. Every row's `attributes` should carry the identity block + (`app.version`, `app.entrypoint`, `user.account_uuid`, `organization.id`, + `terminal.type`). Pass `--max-bytes 0` or the display truncates the JSON and + you will read a short value as a missing one. + +9. Confirm the capture-health line agrees, which is the production half of the + same duty: + + ```sh + hyp status + ``` + + Pass condition: a `capture health:` block with a `- claude last event + ago, last transcript activity ago` line, the two ages + within a few minutes of each other, and **no** `[capture gap]` tag or + `capture_gap` diagnostic. + +10. Record in the release notes: the `claude --version` you ran against, the + full event-name list from step 7, the body top-level keys from step 3, and + any field from steps 4, 6, or 8 that came back null. Those four items are + the release-to-release diff that makes upstream drift visible; a bare + "passed" makes the next run start from nothing. + +### If it fails + +- Step 1 refuses the attach with an upgrade hint: the installed Claude Code is + below 2.1.193. Run `claude update` and start again. The refusal is correct + behavior, not a bug: any existing attach was left byte-for-byte alone + ([LLP 0258#version-floor](../llp/0258-attach-injects-telemetry-via-settings-env.decision.md#version-floor)). +- Step 8 finds `decision` set but `source` null on a Claude Code between + 2.1.193 and 2.1.214: that is the documented gap, not drift. Upgrade and + re-run rather than filing it. +- Step 3 finds an empty spool: check that the conversation ran in a session + started **after** the attach (the settings `env` applies at launch), then + check `jq '.env.OTEL_LOG_RAW_API_BODIES' "$SETTINGS"` names the + spool with the `file:` prefix. If both hold, `OTEL_LOG_RAW_API_BODIES` is no + longer honored upstream. That is the flag-stability failure LLP 0262 open + question 5 predicts. File it and hold the release: events alone lose + `system_text`, the `tools` list, and untruncated tool args. +- Step 4 finds a body whose keys have changed shape: file it with the observed + key list before release and do not paper over it in the projector. The rows + will keep landing with the affected columns null, which is exactly the silent + loss this step exists to make loud. +- Step 5 finds the spool growing rather than draining: the listener is not + consuming. Check `hyp status` for a `@hypaware/claude` source error, confirm + the daemon restarted after step 4, and confirm the port in + `OTEL_EXPORTER_OTLP_ENDPOINT` is the one the listener actually bound (a + dynamic port moves across daemon restarts; `hyp attach claude` rewrites it). +- Step 6 finds rows with `with_system = 0` and `with_tools = 0` while step 4 + passed: the bodies are being written but not joined. Check whether the body + files are landing somewhere other than the attach-written spool, since a + `body_ref` outside the spool is refused by containment and counted, not read. +- Step 7 finds no rows at all while step 6 found messages: the logs exporter is + arriving and the metrics exporter is not, or vice versa. Check + `OTEL_METRICS_EXPORTER` in the env block before suspecting the dataset. +- Step 9 shows `[capture gap]` right after a healthy step 6: the transcript + probe sees session files newer than the last event, usually because the + daemon was down for part of the run. Re-run steps 5 and 9 against a daemon + that stayed up before filing anything. + +--- + ## Other candidates `CLAUDE.md` lists further acceptance candidates that have no written diff --git a/docs/PRIVACY.md b/docs/PRIVACY.md index 1e3f10ec..e20634c9 100644 --- a/docs/PRIVACY.md +++ b/docs/PRIVACY.md @@ -23,6 +23,26 @@ text, not just metadata. Rows age out of the local cache after the retention window init set (90 days on a team install, 120 on a local-only one; `hyp init --retention-days ` overrides). +### The raw-body spool + +With Claude Code attached, Claude Code writes each raw request and response +body into `~/.hyp/spool/claude-bodies`, a directory HypAware creates +owner-only (`0700`). It is a transit area, not storage: HypAware reads a file +only for the few fields its event stream leaves out (the system prompt, the +tool list, message ordering, untruncated tool arguments) and deletes the file +as soon as it has them. The same content is already in Claude Code's own +transcripts under `~/.claude/projects`. + +Three things keep it from becoming a second record: + +- A session you ignored, by `.hypignore`, by a machine-local marking, or with + `hyp session ignore`, has its bodies **deleted unread**, not skipped. +- The directory has a size cap (512 MB by default, `spool_max_bytes` in the + `@hypaware/claude` config). Past it the oldest files go first, so a stopped + daemon costs detail, never disk. +- `hyp purge` empties it, whatever else you asked that purge to delete, and + `hyp detach claude` empties it on the way out. + ### If you turned on proxy mode Proxy mode (see the README) routes all of Claude Code's HTTPS through the @@ -150,13 +170,17 @@ Two caveats apply to both surfaces: To keep one conversation out of the record without marking any directory, run `hyp session ignore` from inside that Claude Code or Codex session. It resolves the session id itself and refuses rather than guessing when it -cannot. Reverse it with `hyp session unignore`; `hyp session status` reports -which state the session is in right now. +cannot, and it posts the opt-out to every local recorder hosting the control +route: the gateway, and the Claude telemetry listener when one is running. +On the listener, a dropped session's spooled raw bodies are deleted, not +merely skipped. Reverse it with `hyp session unignore`; `hyp session status` +reports which state the session is in right now. The opt-out is in-memory and lasts for that session only. Two things drop it -while you may still believe it holds: a gateway restart, and a fork -(`claude --fork-session`, `codex fork`), which mints a new session id the -opt-out no longer covers. A plain resume reuses the id. +while you may still believe it holds: a daemon restart (which drops both +recorders' sets), and a fork (`claude --fork-session`, `codex fork`), which +mints a new session id the opt-out no longer covers. A plain resume reuses +the id. ## Deleting what was already recorded @@ -173,6 +197,11 @@ hyp purge --all # everything, wholesale It prompts on a TTY; pass `--yes` for non-interactive use. +Every form of it also empties the raw-body spool described above, including +the targeted ones: a spooled body has not been read yet, so nothing about it +says which directory or session it belongs to, and leaving it would let the +next batch write back rows you just deleted. + ## Enrolling with a team: the first-sync review Enrollment never ships history silently. When `hyp remote login` (or diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/api.js b/hypaware-core/plugins-workspace/ai-gateway/src/api.js index af2400d2..7e16d7a1 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/api.js +++ b/hypaware-core/plugins-workspace/ai-gateway/src/api.js @@ -1,7 +1,9 @@ // @ts-check +import { createProjectedExchangeWriter } from './exchange_writer.js' + /** - * @import { AiGatewayCapability, AiGatewayEndpointOptions } from '../../../../hypaware-plugin-kernel-types.js' + * @import { AiGatewayCapability, AiGatewayEndpointOptions, AiGatewayProjectedExchange, AiGatewayRecordOptions, QueryStorageService } from '../../../../hypaware-plugin-kernel-types.js' * @import { GatewayState } from './types.js' */ @@ -32,11 +34,19 @@ export function createGatewayState() { * adapter should hand to the client tool so its traffic flows through * this gateway. * + * `storage` is the activation context's storage service. It is what + * lets `recordProjectedExchange` exist: a producer plugin holding a + * finished projection can hand it back to the dataset's owner instead + * of learning the table path, the column list, and the dedupe rules. + * * @param {GatewayState} state + * @param {{ storage?: QueryStorageService }} [deps] * @returns {AiGatewayCapability} */ -export function createAiGatewayApi(state) { +export function createAiGatewayApi(state, deps = {}) { let projectorSeq = 0 + /** @type {ReturnType | undefined} */ + let writer return { registerUpstreamPreset(preset) { if (!preset || typeof preset.name !== 'string' || preset.name.length === 0) { @@ -96,6 +106,28 @@ export function createAiGatewayApi(state) { state.enrichers.set(enricher.clientName, enricher) }, + /** + * Record one already-projected exchange into `ai_gateway_messages`. + * + * The caller supplies the projection; everything downstream of it + * (row expansion, `part_id` identity, the `part_id` dedupe against + * committed and spooled rows, the table path, the column list) stays + * here, so a second live producer cannot drift from the proxy's rows + * for the same content. + * + * @ref LLP 0252#projection-unchanged [implements]: the OTEL listener is a + * third producer of this dataset, not the owner of a new one + * @param {AiGatewayProjectedExchange} projection + * @param {AiGatewayRecordOptions} [opts] + */ + async recordProjectedExchange(projection, opts) { + if (!deps.storage) { + throw new Error('ai-gateway: recordProjectedExchange() needs a storage service') + } + if (!writer) writer = createProjectedExchangeWriter({ storage: deps.storage }) + return writer.record(projection, opts ?? {}) + }, + /** * Resolve the local endpoint URL the gateway is listening on. The * source must be started (`state.listen` set) before this returns diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/dataset.js b/hypaware-core/plugins-workspace/ai-gateway/src/dataset.js index e0611694..df30050a 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/dataset.js +++ b/hypaware-core/plugins-workspace/ai-gateway/src/dataset.js @@ -504,6 +504,52 @@ async function dedupeByPartId(rows, ctx) { return fresh } +/** + * Pre-write `part_id` dedupe for a LIVE producer that is not the proxy + * recorder: the OTEL telemetry listener of `@hypaware/claude`. Same + * membership question the backfill materializer asks, with the same two + * seeds (committed partitions plus the spool), but restricted to the + * keys of the batch in hand so a per-exchange call stays O(batch). + * + * Folding the spool in is safe here and required: the rows being tested + * have NOT been spooled yet, so a spool hit means another producer + * (the proxy, or a backfill run) already wrote this part. That is the + * whole overlap story of the migration window. The hazard note on + * `scanSpooledPartIds` applies to the FLUSH path only, which passes + * rows that are themselves the spool. + * + * Best-effort with respect to storage, like every other dedupe here: a + * stub without the read surface lets every row through. + * + * @ref LLP 0252#projection-unchanged [implements]: a third producer writes the + * same dataset and its overlap with the proxy and backfill producers collapses + * on `part_id` before the write, not after + * @param {Record[]} rows + * @param {QueryStorageService | undefined} storage + * @returns {Promise[]>} + */ +export async function dedupeStoredPartIds(rows, storage) { + if (rows.length === 0 || !canScanExistingRows(storage)) return rows + /** @type {Set} */ + const batchKeys = new Set() + for (const row of rows) { + const key = partIdKey(row) + if (key !== undefined) batchKeys.add(key) + } + const seen = await scanExistingPartIds(storage, batchKeys) + await scanSpooledPartIds(storage, seen, batchKeys) + /** @type {Record[]} */ + const fresh = [] + for (const row of rows) { + const key = partIdKey(row) + if (key === undefined) { fresh.push(row); continue } + if (seen.has(key)) continue + seen.add(key) + fresh.push(row) + } + return fresh +} + /** @param {Record} row */ function isFallbackRow(row) { const attrs = row?.attributes @@ -718,16 +764,24 @@ async function scanExistingPartIds(storage, restrictTo) { * "backfill-vs-spool same-id duplicates" residue by scanning spooled * rows in the materializer (not the settle path). * + * `restrictTo`, when supplied, keeps only the keys of the batch in hand + * (the live-producer caller, `dedupeStoredPartIds`); backfill omits it + * because its per-run memo legitimately needs every spooled key. + * * @param {QueryStorageService} storage * @param {Set} seen + * @param {ReadonlySet} [restrictTo] * @returns {Promise} */ -async function scanSpooledPartIds(storage, seen) { +async function scanSpooledPartIds(storage, seen, restrictTo) { if (!canScanSpooledRows(storage)) return + if (restrictTo && restrictTo.size === 0) return try { for await (const row of storage.readSpooledRows(DATASET_NAME, ['part_id', 'message_id', 'part_index'])) { const key = partIdKey(row) - if (key !== undefined) seen.add(key) + if (key === undefined) continue + if (restrictTo && !restrictTo.has(key)) continue + seen.add(key) } } catch { // Spool unreadable mid-scan: keep whatever we folded in already. diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/exchange_writer.js b/hypaware-core/plugins-workspace/ai-gateway/src/exchange_writer.js new file mode 100644 index 00000000..e602c346 --- /dev/null +++ b/hypaware-core/plugins-workspace/ai-gateway/src/exchange_writer.js @@ -0,0 +1,68 @@ +// @ts-check + +import { + AI_GATEWAY_SCHEMA_COLUMNS, + aiGatewayTablePath, + dedupeStoredPartIds, +} from './dataset.js' +import { + aiGatewayRowsFromProjectedExchange, + createAiGatewayConversationState, +} from './message_projector.js' + +/** + * @import { AiGatewayProjectedExchange, AiGatewayRecordOptions, AiGatewayRecordResult, QueryStorageService } from '../../../../hypaware-plugin-kernel-types.js' + */ + +/** + * The write half of the `ai_gateway_messages` producer contract, for a + * live producer that is not the proxy recorder. + * + * The proxy has a recorder, a wire exchange, and a projector chain in + * front of it; a producer that already holds a finished + * `AiGatewayProjectedExchange` (the Claude OTEL telemetry listener) + * needs only the last two steps, and must not reimplement them: row + * expansion, `part_id` identity, the schema strip, the table path, and + * the dedupe are the dataset owner's business, not the producer's. + * + * One conversation state is held for the process lifetime, exactly as + * the live projector holds one per listener: it is what makes a + * re-delivered event collapse instead of appending a second copy, and + * what threads `previous_message_id` across calls within a session. + * + * @ref LLP 0252#projection-unchanged [implements]: OTEL is a third producer of + * the same dataset, so it enters through the same expansion and dedupe as the + * proxy and the backfill materializer + * @param {{ storage: QueryStorageService, gatewayId?: string }} opts + */ +export function createProjectedExchangeWriter(opts) { + const { storage, gatewayId } = opts + const state = createAiGatewayConversationState() + /** @type {string | undefined} */ + let tablePath + + return { + /** + * Expand one projection into rows, drop the parts some other + * producer already stored, and append what is left. + * + * @param {AiGatewayProjectedExchange} projection + * @param {AiGatewayRecordOptions} [recordOpts] + * @returns {Promise} + */ + async record(projection, recordOpts = {}) { + const rows = aiGatewayRowsFromProjectedExchange(projection, { + ...(gatewayId ? { gatewayId } : {}), + ...(recordOpts.gatewayAttributes ? { gatewayAttributes: recordOpts.gatewayAttributes } : {}), + state, + }) + if (rows.length === 0) return { rowsWritten: 0, rowsSkipped: 0 } + const fresh = await dedupeStoredPartIds(rows, storage) + if (fresh.length > 0) { + if (tablePath === undefined) tablePath = aiGatewayTablePath(storage) + await storage.appendRows(tablePath, [...AI_GATEWAY_SCHEMA_COLUMNS], fresh) + } + return { rowsWritten: fresh.length, rowsSkipped: rows.length - fresh.length } + }, + } +} diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/index.js b/hypaware-core/plugins-workspace/ai-gateway/src/index.js index af4c4645..3f7702dd 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/index.js +++ b/hypaware-core/plugins-workspace/ai-gateway/src/index.js @@ -41,7 +41,7 @@ const PLUGIN_NAME = '@hypaware/ai-gateway' */ export async function activate(ctx) { const state = createGatewayState() - const api = createAiGatewayApi(state) + const api = createAiGatewayApi(state, { storage: ctx.storage }) ctx.provideCapability('hypaware.ai-gateway', '2.0.0', api) ctx.query.registerDataset(aiGatewayDatasetRegistration(state)) @@ -55,15 +55,18 @@ export async function activate(ctx) { start: createStartSource(state), }) - // @ref LLP 0067#cli [implements]: the gateway owns `/_hypaware/ignore/session`, - // so it owns the verbs over it (LLP 0003). One client-agnostic verb group - // serves Claude and Codex alike, and as a plugin-contributed group it - // inherits the inactive-plugin `repair:` line (LLP 0153/0154) for free. - // Deliberately NOT `hyp ignore --session`: LLP 0110 diagnosed that shape. + // @ref LLP 0067#cli [implements]: the gateway hosts the original + // `/_hypaware/ignore/session`, so it owns the verbs over it (LLP 0003). One + // client-agnostic verb group serves Claude and Codex alike, and as a + // plugin-contributed group it inherits the inactive-plugin `repair:` line + // (LLP 0153/0154) for free. The mutations also address every OTHER recorder + // that advertises the route (the claude telemetry listener, LLP 0256), so + // one verb reaches them all. Deliberately NOT `hyp ignore --session`: + // LLP 0110 diagnosed that shape. ctx.commands.register({ name: 'session ignore', plugin: PLUGIN_NAME, - summary: 'Stop recording this AI session (in-memory, until the gateway restarts)', + summary: 'Stop recording this AI session on every local recorder (in-memory, until the daemon restarts)', usage: 'hyp session ignore [session-id] [--json]', run: runSessionIgnore, }) diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/proxy.js b/hypaware-core/plugins-workspace/ai-gateway/src/proxy.js index 1777b5b7..97149d9b 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/proxy.js +++ b/hypaware-core/plugins-workspace/ai-gateway/src/proxy.js @@ -4,10 +4,13 @@ import http from 'node:http' import https from 'node:https' import tls from 'node:tls' +import { isControlPath } from '../../../../src/core/control/session_ignore.js' import { parseListen } from './config.js' import { attachConnectFrontDoor, connectHostOf, connectPortOf, isLoopbackAddress, openUpstream } from './connect.js' import { createNullExchange } from './recorder.js' +export { isControlPath } + /** * @import { AiGatewayRouteInput } from '../../../../hypaware-plugin-kernel-types.js' * @import { CompiledUpstream, ProxyOptions, StartedProxy, UpstreamConfig, UpstreamProxy } from './types.js' @@ -557,19 +560,6 @@ function buildRouteInput(method, pathname, headers) { return { method, path: pathname, headers: flatHeaders } } -/** - * Recognize the reserved `/_hypaware/` local control prefix. Uses the same - * segment-boundary discipline as `pathMatchesPrefix`: `/_hypaware` itself - * and any `/_hypaware/...` sub-path match, but `/_hypawarefoo` does not, so - * a look-alike upstream path is never mistaken for a control request. - * - * @ref LLP 0066#control-path [implements] - * @param {string} pathname - */ -export function isControlPath(pathname) { - return pathname === '/_hypaware' || pathname.startsWith('/_hypaware/') -} - /** * Path-segment prefix match. `/v1/messages` matches `/v1/messages` and * `/v1/messages/anything`, but not `/v1/messagesfoo`. A `/` prefix is diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/session_command.js b/hypaware-core/plugins-workspace/ai-gateway/src/session_command.js index 5acbbe1e..47c7385b 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/session_command.js +++ b/hypaware-core/plugins-workspace/ai-gateway/src/session_command.js @@ -8,12 +8,16 @@ import path from 'node:path' import { readRolloutSessionMeta } from '../../../../src/core/codex/rollout_session_meta.js' import { configuredGatewayEndpoint } from '../../../../src/core/config/gateway_endpoint.js' -import { resolveLiveGatewayEndpointFromStatus } from '../../../../src/core/daemon/status.js' +import { SESSION_IGNORE_ROUTE } from '../../../../src/core/control/session_ignore.js' +import { + resolveLiveControlRouteEndpointsFromStatus, + resolveLiveGatewayEndpointFromStatus, +} from '../../../../src/core/daemon/status.js' import { readObservabilityEnv } from '../../../../src/core/observability/env.js' /** * @import { CommandRunContext } from '../../../../hypaware-plugin-kernel-types.js' - * @import { SessionEndpointResolution, SessionIdResolution, SessionStatusReport } from './types.js' + * @import { SessionEndpointResolution, SessionIdResolution, SessionMutationOutcome, SessionStatusReport } from './types.js' */ const CONTROL_PATH = '/_hypaware/ignore/session' @@ -140,7 +144,7 @@ const MAX_ROLLOUT_SCAN = 5000 * Hard cap on a control response body. The route's own answers are a few dozen * bytes; anything larger is not the gateway, and buffering it unbounded lets * whatever owns the port balloon the CLI's memory. Mirrors the server-side - * `MAX_BODY_BYTES` in control.js. + * `MAX_BODY_BYTES` in src/core/control/session_ignore.js. */ const MAX_RESPONSE_BYTES = 64 * 1024 @@ -292,9 +296,24 @@ export async function runSessionStatus(argv, ctx) { } /** - * Shared body for `ignore` / `unignore`: resolve the id, resolve the - * endpoint, then toggle. Mutations fail closed the same way `status` does - - * an unreachable gateway is an error, never a quiet success. + * Shared body for `ignore` / `unignore`: resolve the id, resolve every + * recorder hosting the control route, then toggle on each. Mutations fail + * closed the same way `status` does - an unreachable recorder is an error, + * never a quiet success - and a PARTIAL success is reported as partial, so + * "the gateway took it but the claude listener refused" never reads as done. + * + * Two kinds of recorder, resolved differently on purpose: + * + * - **The gateway** keeps its own two-rung resolution (live daemon status, + * else the pinned `listen`), unchanged, because it can run outside a + * daemon this command can see. + * - **Additional recorders** (the claude telemetry listener) are discovered + * by the `control_routes` advertisement in a LIVE daemon snapshot. One + * that does not appear there is not running, and a listener that is not + * running is recording nothing, so it is not addressed and its absence is + * not a failure. One that IS addressed and refuses is. + * @ref LLP 0256#cli-posts-to-both [implements]: every listener that offers + * the route is addressed, each outcome is reported, partial is not swallowed * * @param {string[]} argv * @param {CommandRunContext} ctx @@ -317,44 +336,94 @@ async function runMutation(argv, ctx, method, usage) { return SESSION_EXIT_UNKNOWN } - const endpoint = resolveGatewayEndpointForCli(ctx) - if (!endpoint.ok) { - ctx.stderr.write(`hyp session: ${endpoint.error}\n`) + const gateway = resolveGatewayEndpointForCli(ctx) + const advertised = resolveAdvertisedRecordersForCli(ctx, gateway.ok ? gateway.endpoint : undefined) + + // NO recorder at all is the old no-gateway error: nothing would hold the + // token, so nothing may read as success. + if (!gateway.ok && advertised.length === 0) { + ctx.stderr.write(`hyp session: ${gateway.error}\n`) return SESSION_EXIT_UNKNOWN } + // A resolvable extra recorder with no resolvable gateway is possible only + // when the live snapshot carries the listener but no bound gateway port: + // the gateway is not listening, so it records nothing and is not + // addressed - said out loud rather than silently narrowed. + if (!gateway.ok) { + ctx.stderr.write(`hyp session: gateway not addressed: ${gateway.error}\n`) + } - const result = await controlRequest({ - endpoint: endpoint.endpoint, - method, - sessionId: resolvedId.sessionId, - }) - if (!result.ok) { - ctx.stderr.write(`hyp session: ${result.error}\n`) - return SESSION_EXIT_UNKNOWN + /** @type {Array<{ recorder: string, endpoint: string, endpointSource: 'daemon_status' | 'config_listen' }>} */ + const targets = [] + if (gateway.ok) targets.push({ recorder: 'gateway', endpoint: gateway.endpoint, endpointSource: gateway.source }) + for (const extra of advertised) { + targets.push({ recorder: extra.source, endpoint: extra.endpoint, endpointSource: 'daemon_status' }) } - const ignored = result.body.ignored - const total = result.body.total + /** @type {SessionMutationOutcome[]} */ + const outcomes = [] + for (const target of targets) { + const result = await controlRequest({ + endpoint: target.endpoint, + method, + sessionId: resolvedId.sessionId, + }) + outcomes.push( + result.ok + ? { ...target, ok: true, ignored: result.body.ignored, total: result.body.total } + : { ...target, ok: false, error: result.error } + ) + } + + // Each failure is reported next to the successes, never instead of them: + // the failed recorder is the one still recording, which is exactly what + // the user asked to stop. + for (const outcome of outcomes) { + if (!outcome.ok) { + ctx.stderr.write(`hyp session: ${outcome.recorder} at ${outcome.endpoint}: ${outcome.error}\n`) + } + } + const confirmed = outcomes.filter((o) => o.ok) + if (confirmed.length === 0) return SESSION_EXIT_UNKNOWN + const allOk = confirmed.length === outcomes.length + + // The first confirmed recorder (the gateway whenever it was addressed) + // keeps the legacy top-level receipt shape; every recorder's outcome rides + // beside it, so a consumer of the old fields loses nothing and a consumer + // of the new field sees the whole write. + const primary = confirmed[0] if (parsed.json) { ctx.stdout.write( JSON.stringify({ - status: 'ok', + // `partial` and exit UNKNOWN when an addressed recorder refused: an + // `ok` over a recorder that is still recording would be the + // fail-open receipt this verb exists to prevent. + status: allOk ? 'ok' : 'partial', // What the `ok` above is an `ok` about, for the agent parsing this. guarantee: MEMBERSHIP_GUARANTEE, session_id: resolvedId.sessionId, session_id_source: resolvedId.source, session_id_evidence: resolvedId.evidence ?? null, thread_id: resolvedId.threadId ?? null, - ignored, - total, - endpoint: endpoint.endpoint, - endpoint_source: endpoint.source, + ignored: primary.ignored, + total: primary.total, + endpoint: primary.endpoint, + endpoint_source: primary.endpointSource, // Same field, same constant, on the verbs whose output reads as done. // @ref LLP 0166#stated-not-proved [implements] endpoint_authenticated: false, + recorders: outcomes.map((o) => ({ + recorder: o.recorder, + endpoint: o.endpoint, + endpoint_source: o.endpointSource, + endpoint_authenticated: false, + ...(o.ok + ? { status: 'ok', ignored: o.ignored, total: o.total } + : { status: 'error', error: o.error }), + })), }) + '\n' ) - return 0 + return allOk ? 0 : SESSION_EXIT_UNKNOWN } // The headline states the write that happened, not a drop nobody verified: // the route added an opaque token to a set (LLP 0066#receipt-is-membership). @@ -362,11 +431,21 @@ async function runMutation(argv, ctx, method, usage) { // overclaim mirrored: a token nothing carried suppressed nothing to resume, // and the folder governor below is a separate reason a session stays unrecorded. ctx.stdout.write( - ignored - ? `session ${resolvedId.sessionId}: ignored - this id is in the gateway drop set (${total} ignored)\n` - : `session ${resolvedId.sessionId}: not ignored - this id is out of the gateway drop set, so this opt-out suppresses nothing now (${total} ignored)\n` + primary.ignored + ? `session ${resolvedId.sessionId}: ignored - this id is in the ${primary.recorder} drop set (${primary.total} ignored)\n` + : `session ${resolvedId.sessionId}: not ignored - this id is out of the ${primary.recorder} drop set, so this opt-out suppresses nothing now (${primary.total} ignored)\n` ) - if (ignored) { + // Every further confirmed recorder gets its own line: "ignored" on one + // recorder is not ignored while a second one records, so each write is + // named rather than folded into the headline. + for (const outcome of confirmed.slice(1)) { + ctx.stdout.write( + outcome.ignored + ? `also ${outcome.recorder} at ${outcome.endpoint}: ignored - this id is in its drop set (${outcome.total} ignored)\n` + : `also ${outcome.recorder} at ${outcome.endpoint}: not ignored - this id is out of its drop set (${outcome.total} ignored)\n` + ) + } + if (primary.ignored) { ctx.stdout.write(`${EPHEMERAL_NOTE}\n`) ctx.stdout.write(`${MEMBERSHIP_NOTE}\n`) } @@ -377,13 +456,46 @@ async function runMutation(argv, ctx, method, usage) { idSource: resolvedId.source, idEvidence: resolvedId.evidence ?? null, threadId: resolvedId.threadId ?? null, - endpoint: endpoint.endpoint, - endpointSource: endpoint.source, + endpoint: primary.endpoint, + endpointSource: primary.endpointSource, })) { ctx.stdout.write(`${note}\n`) } + // The trust contract is per responder, and it is unconditional (LLP 0166), + // so each further endpoint gets the same disclosure the primary one got. + for (const outcome of confirmed.slice(1)) { + ctx.stdout.write(`${responderTrustNote(outcome.endpoint)}\n`) + } ctx.stdout.write(`${FOLDER_GOVERNOR_NOTE}\n`) - return 0 + return allOk ? 0 : SESSION_EXIT_UNKNOWN +} + +/** + * The recorders beyond the gateway that host the session-ignore control + * route, discovered by their own `control_routes` advertisement in a live + * daemon snapshot (`resolveLiveControlRouteEndpointsFromStatus`). + * + * An advertised endpoint equal to the gateway's is dropped: the gateway is + * already a target through its own resolution, and posting one mutation + * twice would double every log line and confuse the receipt. Errors reading + * the snapshot resolve to "no additional recorders", which is exact: with + * no live snapshot there is no daemon, and these listeners only run inside + * one. + * + * @param {CommandRunContext} ctx + * @param {string | undefined} gatewayEndpoint + * @returns {Array<{ source: string, endpoint: string }>} + */ +function resolveAdvertisedRecordersForCli(ctx, gatewayEndpoint) { + /** @type {Array<{ source: string, endpoint: string }>} */ + let list + try { + const stateRoot = readObservabilityEnv(ctx.env).stateDir + list = resolveLiveControlRouteEndpointsFromStatus({ stateRoot, route: SESSION_IGNORE_ROUTE }) + } catch { + return [] + } + return list.filter((entry) => entry.endpoint !== gatewayEndpoint) } /** diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/source.js b/hypaware-core/plugins-workspace/ai-gateway/src/source.js index 9449b687..94746945 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/source.js +++ b/hypaware-core/plugins-workspace/ai-gateway/src/source.js @@ -16,7 +16,7 @@ import { } from '../../../../src/core/tls/ca.js' import { compileConfig, compileUpstreams, FALLBACK_LISTEN } from './config.js' -import { createControlHandler } from './control.js' +import { createControlHandler } from '../../../../src/core/control/session_ignore.js' import { AI_GATEWAY_SCHEMA_COLUMNS, aiGatewayTablePath, DATASET_NAME } from './dataset.js' import { createEntrypointActivity } from './entrypoint_activity.js' import { createAiGatewayMessageProjector } from './message_projector.js' diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/types.d.ts b/hypaware-core/plugins-workspace/ai-gateway/src/types.d.ts index 3c1e5f8c..72045307 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/types.d.ts +++ b/hypaware-core/plugins-workspace/ai-gateway/src/types.d.ts @@ -254,6 +254,23 @@ export type SessionEndpointResolution = | { ok: true; endpoint: string; source: 'daemon_status' | 'config_listen' } | { ok: false; error: string } +/** + * One recorder's outcome for a `hyp session ignore` / `unignore` write. The + * verb addresses every recorder hosting the control route (the gateway plus + * whatever a live daemon snapshot advertises, LLP 0256 #cli-posts-to-both) + * and reports each outcome; `ignored` / `total` are present exactly when + * `ok` is true, `error` exactly when it is not. + */ +export interface SessionMutationOutcome { + recorder: string + endpoint: string + endpointSource: 'daemon_status' | 'config_listen' + ok: boolean + ignored?: boolean + total?: number + error?: string +} + /** * What `hyp session status` reports, in `--json` field order. It carries the * PROVENANCE of both inputs alongside the answer (`session_id_source` / diff --git a/hypaware-core/plugins-workspace/claude/hypaware.plugin.json b/hypaware-core/plugins-workspace/claude/hypaware.plugin.json index 296f70da..5a40c650 100644 --- a/hypaware-core/plugins-workspace/claude/hypaware.plugin.json +++ b/hypaware-core/plugins-workspace/claude/hypaware.plugin.json @@ -8,6 +8,7 @@ "node_engine": ">=20", "entrypoint": "./src/index.js", "permissions": [ + "network", "read_home", "write_home", "read_state", @@ -20,6 +21,14 @@ } }, "contributes": { + "sources": [{ "name": "claude-telemetry" }], + "datasets": [ + { + "name": "claude_telemetry_events", + "source": "claude-telemetry", + "summary": "Claude Code behavioral telemetry: one row per event (tool decisions, permission mode changes, hook and MCP health, cost and activity metrics)" + } + ], "client": { "name": "claude", "skill_dir": ".claude/skills", @@ -29,6 +38,10 @@ "settings_file": ".claude/settings.json", "marker_key": "_hypaware" }, + "activity_probe": { + "dir": ".claude/projects", + "file_suffix": ".jsonl" + }, "required_upstreams": ["anthropic"], "transcript_entrypoints": ["cli", "sdk-cli"], "launch": { "bin": "claude", "args": ["{prompt}"], "label": "Claude Code" } @@ -64,7 +77,7 @@ "config_sections": [ { "section": "claude", - "summary": "Claude adapter config, including the optional backfill-on-join policy { on_join, window_days }." + "summary": "Claude adapter config: the optional backfill-on-join policy { on_join, window_days }, the attach-on-join policy { on_join }, and the telemetry listener address { listen_host, listen_port }." } ] } diff --git a/hypaware-core/plugins-workspace/claude/skills/hypaware-reference/SKILL.md b/hypaware-core/plugins-workspace/claude/skills/hypaware-reference/SKILL.md index bc3bbfc6..3a247644 100644 --- a/hypaware-core/plugins-workspace/claude/skills/hypaware-reference/SKILL.md +++ b/hypaware-core/plugins-workspace/claude/skills/hypaware-reference/SKILL.md @@ -86,11 +86,12 @@ curated HypAware registry. with no repo breadcrumb. - Stop recording *this conversation* - `hyp session ignore` drops this session's - exchanges at the gateway; `hyp session unignore` resumes, and `hyp session - status` reports which it is right now. Each resolves the session id itself - (Claude and Codex) and fails closed rather than guessing. The opt-out is - in-memory: a gateway restart drops it, and a fork (`claude --fork-session`, - `codex fork`) mints a new id it no longer covers. + exchanges at every local recorder (the gateway, and the Claude telemetry + listener when one is running); `hyp session unignore` resumes, and + `hyp session status` reports which it is right now. Each resolves the + session id itself (Claude and Codex) and fails closed rather than guessing. + The opt-out is in-memory: a daemon restart drops it, and a fork + (`claude --fork-session`, `codex fork`) mints a new id it no longer covers. - Decide what happens in new folders - by default they sync with no question; `hyp policy folders ask` asks once per new folder instead, and `hyp policy folders sync` returns to the default. It gates the question diff --git a/hypaware-core/plugins-workspace/claude/src/claude_version.js b/hypaware-core/plugins-workspace/claude/src/claude_version.js new file mode 100644 index 00000000..6051ff7a --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/claude_version.js @@ -0,0 +1,138 @@ +// @ts-check + +import { execFile } from 'node:child_process' +import { promisify } from 'node:util' + +const execFileAsync = promisify(execFile) + +/** + * The installed Claude Code release, and the floor `otel` attach needs. + * + * OTEL attach is the only mechanism the `claude` client has, so the floor is + * enforced at attach rather than discovered later as an empty dataset: a + * release below it emits none of the events the listener reads, and the user + * would be left with a settings file that says "attached" and a capture that + * never starts. + * + * @ref LLP 0258#version-floor [implements]: the floor the attach adapter checks + * before it is allowed to switch the client to `otel` mode + */ + +/** First Claude Code release that emits the telemetry event set. */ +export const CLAUDE_OTEL_MIN_VERSION = '2.1.193' + +/** + * First release that carries `tool_source` on tool-decision events. Above the + * floor, so it never blocks an attach: a machine between the two captures + * everything except which surface approved a tool call. + */ +export const CLAUDE_TOOL_SOURCE_MIN_VERSION = '2.1.214' + +/** What the user runs to clear the floor. Kept as one string so every surface prints the same hint. */ +export const CLAUDE_UPDATE_HINT = 'claude update' + +/** + * Pull a dotted release number out of whatever `claude --version` prints. + * The current format is `2.1.233 (Claude Code)`, but only the leading numeric + * triple is contractual enough to depend on, so everything after it is + * ignored rather than matched. + * + * @param {unknown} text + * @returns {string | undefined} the version, or `undefined` when none is readable + */ +export function parseClaudeVersion(text) { + if (typeof text !== 'string') return undefined + const match = /(\d+)\.(\d+)\.(\d+)/.exec(text) + return match ? `${match[1]}.${match[2]}.${match[3]}` : undefined +} + +/** + * Compare two dotted release numbers numerically, not lexically: `2.1.193` + * sorts *below* `2.1.9` under a string compare, which would refuse exactly the + * releases that clear the floor. + * + * @param {string} a + * @param {string} b + * @returns {number} negative when `a` is older, 0 when equal, positive when newer + */ +export function compareClaudeVersions(a, b) { + const left = a.split('.').map((part) => Number.parseInt(part, 10)) + const right = b.split('.').map((part) => Number.parseInt(part, 10)) + const length = Math.max(left.length, right.length) + for (let i = 0; i < length; i++) { + const l = Number.isInteger(left[i]) ? left[i] : 0 + const r = Number.isInteger(right[i]) ? right[i] : 0 + if (l !== r) return l - r + } + return 0 +} + +/** + * Is this Claude Code demonstrably older than `floor`? + * + * **Unknown is not old.** An undetectable version (`claude` not on PATH, a + * sandboxed attach, a fleet install that renamed the binary) answers `false` + * and the attach proceeds. Refusing on "we could not tell" would turn a + * best-effort probe into a hard dependency on the binary being where we + * looked, and would block the machines most likely to be running a current + * release. Only a version we read and understood can refuse. + * + * @ref LLP 0258#version-floor [constrained-by]: *older than* the floor refuses; + * nothing else does + * @param {string | undefined} version + * @param {string} [floor] + * @returns {boolean} + */ +export function isBelowClaudeVersion(version, floor = CLAUDE_OTEL_MIN_VERSION) { + const parsed = parseClaudeVersion(version) + if (parsed === undefined) return false + return compareClaudeVersions(parsed, floor) < 0 +} + +/** + * Best-effort read of the installed Claude Code version by running + * `claude --version`. + * + * Never throws and never blocks an attach for long: a missing binary, a + * non-zero exit, or a hang all resolve to `undefined`, which + * {@link isBelowClaudeVersion} treats as "not proven old". + * + * @param {{ exec?: typeof execFileAsync, bin?: string, timeoutMs?: number }} [opts] + * `exec` is the subprocess seam, injected in tests so the floor logic is + * testable without a Claude Code install. + * @returns {Promise} + */ +export async function detectClaudeCodeVersion(opts = {}) { + const exec = opts.exec ?? execFileAsync + const bin = opts.bin ?? 'claude' + try { + const result = await exec(bin, ['--version'], { timeout: opts.timeoutMs ?? 3000 }) + return parseClaudeVersion(result?.stdout) + } catch { + return undefined + } +} + +/** + * The version the attach floor checks: the `HYP_CLAUDE_CODE_VERSION` + * environment override when present, otherwise the binary probe. + * + * The override serves two callers with the same need. Hermetic smokes must not + * inherit whatever `claude` the machine running them happens to carry (a stale + * install would flip an unrelated attach smoke to a refusal), and a fleet + * install whose launcher hides the real binary from PATH knows its version + * better than the probe does. An unparseable override falls back to the probe + * rather than silently reading as "unknown, proceed". + * + * @param {NodeJS.ProcessEnv} env + * @param {{ exec?: typeof execFileAsync, bin?: string, timeoutMs?: number }} [opts] + * @returns {Promise} + */ +export async function resolveClaudeCodeVersion(env, opts) { + const override = env.HYP_CLAUDE_CODE_VERSION + if (typeof override === 'string' && override.trim() !== '') { + const parsed = parseClaudeVersion(override) + if (parsed !== undefined) return parsed + } + return detectClaudeCodeVersion(opts) +} diff --git a/hypaware-core/plugins-workspace/claude/src/config.js b/hypaware-core/plugins-workspace/claude/src/config.js index a137461b..d00168a9 100644 --- a/hypaware-core/plugins-workspace/claude/src/config.js +++ b/hypaware-core/plugins-workspace/claude/src/config.js @@ -2,9 +2,11 @@ /** * Config validation for the `@hypaware/claude` plugin's own `config` - * block. v1 validates the optional `backfill` sub-object that drives - * backfill-on-join (`{ on_join, window_days }`), and the optional - * `attach` sub-object that drives attach-on-join, `{ on_join }`. Every + * block. It validates the optional `backfill` sub-object that drives + * backfill-on-join (`{ on_join, window_days }`), the optional `attach` + * sub-object that drives attach-on-join (`{ on_join }`), and the + * optional `telemetry` sub-object that places the Claude telemetry + * listener (`{ listen_host, listen_port }`). Every * other key (e.g. `proxy`) passes through untouched so existing configs * keep working; there is no top-level `backfill`/`attach` section and * nothing new for core to validate. @@ -42,6 +44,7 @@ export function validateClaudeConfig(value) { const errors = [ ...validateBackfillSection(raw.backfill, '/backfill'), ...validateAttachSection(raw.attach, '/attach'), + ...validateTelemetrySection(raw.telemetry, '/telemetry'), ] if (errors.length > 0) return { ok: false, errors } return { ok: true } @@ -88,6 +91,64 @@ export function validateBackfillSection(value, pointer) { return errors } +/** + * Validate the optional `telemetry` block: where the Claude telemetry + * listener binds (`listen_host`, string; `listen_port`, integer in + * `0..65535` where `0` asks for a dynamic port) and how large the raw + * body spool may grow (`spool_max_bytes`, positive integer, default + * 512 MB). All optional; unknown keys are rejected so a typo + * (`listen_ports`) surfaces instead of silently leaving the listener on + * its default port while attach writes the address the operator meant. + * + * @ref LLP 0257#registration [implements]: the listener's port is config with a + * default, and `0` requests a dynamic port + * @ref LLP 0253#byte-cap [implements]: the spool cap is a configured byte value + * + * @param {unknown} value + * @param {string} pointer JSON-pointer prefix for the `telemetry` object + * @returns {ValidationError[]} + */ +export function validateTelemetrySection(value, pointer) { + /** @type {ValidationError[]} */ + const errors = [] + if (value === undefined) return errors + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + errors.push({ pointer, message: 'telemetry must be an object' }) + return errors + } + const raw = /** @type {Record} */ (value) + if (raw.listen_host !== undefined && (typeof raw.listen_host !== 'string' || raw.listen_host.length === 0)) { + errors.push({ + pointer: `${pointer}/listen_host`, + message: 'telemetry.listen_host must be a non-empty string', + }) + } + if (raw.listen_port !== undefined) { + const port = raw.listen_port + if (typeof port !== 'number' || !Number.isInteger(port) || port < 0 || port > 65535) { + errors.push({ + pointer: `${pointer}/listen_port`, + message: 'telemetry.listen_port must be an integer between 0 and 65535', + }) + } + } + if (raw.spool_max_bytes !== undefined) { + const cap = raw.spool_max_bytes + if (typeof cap !== 'number' || !Number.isInteger(cap) || cap < 1) { + errors.push({ + pointer: `${pointer}/spool_max_bytes`, + message: 'telemetry.spool_max_bytes must be a positive integer', + }) + } + } + for (const key of Object.keys(raw)) { + if (key !== 'listen_host' && key !== 'listen_port' && key !== 'spool_max_bytes') { + errors.push({ pointer: `${pointer}/${key}`, message: `unknown telemetry key '${key}'` }) + } + } + return errors +} + /** * Validate the optional `attach` policy block on a client-adapter plugin's * config: `on_join` (whether the daemon auto-attaches this client when a diff --git a/hypaware-core/plugins-workspace/claude/src/hook_command.js b/hypaware-core/plugins-workspace/claude/src/hook_command.js index dc6f0cab..4c82750f 100644 --- a/hypaware-core/plugins-workspace/claude/src/hook_command.js +++ b/hypaware-core/plugins-workspace/claude/src/hook_command.js @@ -4,7 +4,13 @@ import { execFile } from 'node:child_process' import path from 'node:path' import { promisify } from 'node:util' +import { readObservabilityEnv } from '../../../../src/core/observability/env.js' import { appendSessionContext } from './session_context.js' +import { + DEFAULT_SPOOL_MAX_BYTES, + claudeBodySpoolDir, + enforceClaudeBodySpoolCap, +} from './telemetry/spool.js' /** * @import { CommandRunContext } from '../../../../hypaware-plugin-kernel-types.js' @@ -12,6 +18,9 @@ import { appendSessionContext } from './session_context.js' const execFileAsync = promisify(execFile) +/** The plugin whose v2 config slice carries the spool cap this hook applies. */ +const PLUGIN_NAME = '@hypaware/claude' + /** * `hyp claude-hook session-context --state-file ` * @@ -37,21 +46,60 @@ const execFileAsync = promisify(execFile) * least has `cwd`, collapsing the session-start race window from "hook * latency + 2 git execs" to ~one file append. * + * Every invocation also enforces the raw-body spool's byte cap on its way out + * (see {@link sweepBodySpool}), which is the half of LLP 0253's bound the + * daemon cannot deliver on its own. + * * @ref LLP 0085 [implements]: part (a) - shrink the null-cwd window at the * source by writing cwd before the git lookups. * @param {string[]} argv * @param {CommandRunContext} ctx - * @param {{ gitBranch?: typeof currentGitBranch, gitRepoFacts?: typeof gitRepoFacts }} [deps] - * injectable git lookups (tests); default to the real subprocess helpers. + * @param {{ gitBranch?: typeof currentGitBranch, gitRepoFacts?: typeof gitRepoFacts, sweepSpool?: typeof sweepBodySpool }} [deps] + * injectable git lookups and spool sweep (tests); default to the real + * subprocess helpers and the real sweep. */ export async function runClaudeSessionContextHook(argv, ctx, deps = {}) { if (argv.includes('--help') || argv.includes('-h')) { ctx.stdout.write('usage: hyp claude-hook session-context --state-file \n') return 0 } + + // The recording half is already internally fault-tolerant, but it is wrapped + // here too so the invariant holds structurally: whatever it does, the hook + // exits 0 and the sweep below still runs. + try { + await recordSessionContext(argv, ctx, deps) + } catch { + /* hook MUST never throw back into Claude Code */ + } + + // LAST, and on EVERY invocation, including the ones that recorded no + // context. A malformed event or a missing `--state-file` says nothing about + // whether Claude Code is filling the spool, and running last means a slow + // directory can never delay the records above: the projector waits on + // those, nothing waits on this. + try { + await (deps.sweepSpool ?? sweepBodySpool)(ctx) + } catch { + /* a spool that cannot be swept is never worth interrupting Claude for */ + } + return 0 +} + +/** + * Append the session-context record(s) for one hook event. Extracted from + * {@link runClaudeSessionContextHook} so every "nothing to record" exit still + * reaches the spool sweep that follows it. + * + * @param {string[]} argv + * @param {CommandRunContext} ctx + * @param {{ gitBranch?: typeof currentGitBranch, gitRepoFacts?: typeof gitRepoFacts }} deps + * @returns {Promise} + */ +async function recordSessionContext(argv, ctx, deps) { const parsed = parseArgs(argv) const stateFile = parsed.stateFile ?? (parsed.legacyPort ? legacyStateFile(ctx.env) : undefined) - if (!stateFile) return 0 + if (!stateFile) return const input = await readStdin(ctx.stdin ?? process.stdin) /** @type {Record} */ @@ -62,12 +110,12 @@ export async function runClaudeSessionContextHook(argv, ctx, deps = {}) { ? /** @type {Record} */ (parsedEvent) : {} } catch { - return 0 + return } const sessionId = str(event.session_id) const cwd = str(event.new_cwd) ?? str(event.cwd) - if (!sessionId || !cwd) return 0 + if (!sessionId || !cwd) return const transcriptPath = str(event.transcript_path) // Minimal record FIRST: cwd is what the .hypignore policy check needs and it @@ -79,7 +127,7 @@ export async function runClaudeSessionContextHook(argv, ctx, deps = {}) { try { await appendSessionContext(stateFile, /** @type {any} */ (minimal)) } catch { - /* hook MUST never throw back into Claude: exit 0 even on write failure */ + /* hook MUST never throw back into Claude: a write failure records nothing */ } // Enriched record SECOND: run the (slower) git subprocesses, then append the @@ -109,9 +157,89 @@ export async function runClaudeSessionContextHook(argv, ctx, deps = {}) { await appendSessionContext(stateFile, /** @type {any} */ (record)) } } catch { - /* git or write failure: the minimal record already landed; exit 0 */ + /* git or write failure: the minimal record already landed */ } - return 0 +} + +/** + * Enforce the raw-body spool's byte cap from OUTSIDE the daemon. + * + * LLP 0253 #byte-cap names the window the cap exists for: "the window this + * exists for is precisely the one where the reader is not running". Every + * enforcement that decision shipped with, though, lives inside the listener + * source (its one-shot sweep at start and its 60-second timer), so the one + * window it names is the one window nothing swept. Claude Code keeps writing + * bodies whether or not the daemon is up, and the daemon is legitimately down + * for a crashed service, a machine where one was never started, an uninstall + * that skipped detach, and the attach-before-first-start path the port + * resolver deliberately supports. In every one of those the directory grew at + * roughly 145 KB per request with nothing bounding it: unbounded retention of + * raw prompts, not merely a disk nit, because the attach turns + * `OTEL_LOG_USER_PROMPTS` and `OTEL_LOG_ASSISTANT_RESPONSES` on. + * + * The hook is the right second enforcer because it already runs in its own + * process at exactly the cadence bodies are written (SessionStart, CwdChanged, + * UserPromptSubmit, and PostToolUse on Bash: LLP 0085), so the spool cannot + * outrun it, and because it needs nothing the daemon owns. + * + * It deletes only what the daemon's own sweep would have deleted: the same + * `enforceClaudeBodySpoolCap` over the same directory at the same cap, with + * the same oldest-first order. The hook never widens the deletion rule, it + * only runs the existing one while the daemon cannot. + * + * Cost is one `readdir` plus a `stat` per file. On an attached machine with + * the daemon up the directory is near-empty, because the listener deletes what + * it projects; on a proxy-attached or unattached machine it does not exist and + * the sweep returns on `enforceClaudeBodySpoolCap`'s ENOENT arm without a + * single stat. Either way it is well under the two git subprocesses the same + * hook already spawns. + * + * @ref LLP 0263#hook-enforces-the-cap [implements]: the client hook is the + * second enforcer, so the cap holds in the window the daemon is down + * @param {CommandRunContext} ctx + * @returns {Promise} + */ +async function sweepBodySpool(ctx) { + const dir = claudeBodySpoolDir(readObservabilityEnv(ctx.env).hypHome) + await enforceClaudeBodySpoolCap(dir, readSpoolMaxBytes(ctx.config)) +} + +/** + * The cap this hook applies, read from the same `telemetry.spool_max_bytes` + * key the listener's `readSpoolConfig` reads, out of the `@hypaware/claude` + * slice of the v2 config the hook already carries. Matching the listener's + * validation (a positive integer, anything else is the default) is what keeps + * the two enforcers from disagreeing about how large the spool may be. + * + * A bad value falls back silently rather than warning: the listener already + * warns about exactly this key, and a hook has no output surface that would + * not push text at Claude Code. + * + * @ref LLP 0263#hook-enforces-the-cap [constrained-by]: the hook applies the + * operator's cap, never a rule of its own + * @param {unknown} config + * @returns {number} + */ +function readSpoolMaxBytes(config) { + const plugins = obj(config)?.plugins + if (!Array.isArray(plugins)) return DEFAULT_SPOOL_MAX_BYTES + const entry = plugins.find((p) => obj(p)?.name === PLUGIN_NAME) + const raw = obj(obj(obj(entry)?.config)?.telemetry)?.spool_max_bytes + if (typeof raw === 'number' && Number.isInteger(raw) && raw >= 1) return raw + return DEFAULT_SPOOL_MAX_BYTES +} + +/** + * Narrow a value to a plain object, so the config walk above can step through + * a hand-edited config without a `TypeError` reaching Claude Code. + * + * @param {unknown} value + * @returns {Record | undefined} + */ +function obj(value) { + return value && typeof value === 'object' && !Array.isArray(value) + ? /** @type {Record} */ (value) + : undefined } /** diff --git a/hypaware-core/plugins-workspace/claude/src/index.js b/hypaware-core/plugins-workspace/claude/src/index.js index eb712d75..5d6311da 100644 --- a/hypaware-core/plugins-workspace/claude/src/index.js +++ b/hypaware-core/plugins-workspace/claude/src/index.js @@ -10,17 +10,23 @@ import { Attr, getLogger, withSpan } from '../../../../src/core/observability/in import { readObservabilityEnv } from '../../../../src/core/observability/env.js' import { defaultConfigPath } from '../../../../src/core/config/schema.js' import { localOnlyListPath } from '../../../../src/core/usage-policy/index.js' -import { defaultStateRoot, readLocalCaInfo } from '../../../../src/core/tls/ca.js' -import { installCaTrust, isCaTrusted } from '../../../../src/core/tls/darwin_trust.js' -import { installLaunchdEnv } from '../../../../src/core/daemon/launchd_env.js' +import { removeLaunchdEnv } from '../../../../src/core/daemon/launchd_env.js' import { CLAUDE_CONFIG_SECTION, validateClaudeConfig } from './config.js' -import { MODE_BASE_URL, MODE_PROXY, attach, defaultSettingsPath } from './settings.js' +import { MODE_OTEL, MODE_PROXY, attach, defaultSettingsPath } from './settings.js' +import { resolveClaudeCodeVersion } from './claude_version.js' import { anthropicUpstreamPreset, createClaudeExchangeProjector } from './projector.js' import { createClaudeBackfillProvider } from './backfill.js' import { createClaudeSettlementEnricher } from './settle.js' import { defaultSessionContextFile } from './session_context.js' import { runClaudeSessionContextHook } from './hook_command.js' import { runClaudeClassifyHook } from './classify_hook.js' +import { + CLAUDE_TELEMETRY_SOURCE, + createStartClaudeTelemetrySource, + resolveAttachTelemetryPort, +} from './telemetry/source.js' +import { claudeTelemetryDatasetRegistration } from './telemetry/events_dataset.js' +import { claudeBodySpoolDir, ensureClaudeBodySpool } from './telemetry/spool.js' /** * @import { AiGatewayCapability, AiGatewayClientAttachContext, CommandRunContext, HypAwareV2Config, PluginActivationContext } from '../../../../hypaware-plugin-kernel-types.js' @@ -173,86 +179,168 @@ export async function activate(ctx) { dry_run: attachCtx.dryRun === true, }, async (span) => { + const obsEnv = readObservabilityEnv(ctx.env) + // Everything the `otel` env block names, resolved before the + // settings write so the write is one atomic decision. `otel` is the + // claude client's only attach mode: a machine still carrying a + // proxy attach is migrated by this same write (the mode-switch key + // release plus the residue unwind below), never re-attached by + // proxy. + // @ref LLP 0258#version-floor [constrained-by]: one attach mode per client, with no proxy fallback + const telemetryPort = resolveAttachTelemetryPort({ + stateRoot: obsEnv.stateDir, + config: ctx.config, + }) + const spoolDir = claudeBodySpoolDir(obsEnv.hypHome) if (attachCtx.dryRun) { span.setAttribute('status', 'ok') span.setAttribute('restored', false) - const port = safeEndpointPort(attachCtx.endpoint) - const dryRunCa = await readLocalCaInfo({ stateRoot: defaultStateRoot(ctx.env) }) writeAttachOutput(attachCtx, { status: 'ok', client: CLIENT_NAME, dryRun: true, settingsPath, - port, + port: safeEndpointPort(attachCtx.endpoint), changed: false, prevValue: undefined, - mode: dryRunCa ? MODE_PROXY : MODE_BASE_URL, - caCertPath: dryRunCa?.certPath, + mode: MODE_OTEL, + telemetryPort, + spoolDir, }) return } const port = endpointPort(attachCtx.endpoint) try { - // Proxy mode is used when the daemon is actually running it, which - // is exactly when a machine-local CA exists. Reading it here rather - // than from config keeps attach honest: the mode it writes is the - // mode the gateway is serving, not the one someone asked for. - // @ref LLP 0232#proxy-attach-preflight [implements] - const ca = await readLocalCaInfo({ stateRoot: defaultStateRoot(ctx.env) }) + // The floor check itself lives in attach(): it refuses before any + // I/O, so a too-old Claude Code leaves the settings byte-identical, + // a proxy attach it would otherwise have migrated included. + // @ref LLP 0258#version-floor [implements]: the probed version is what attach refuses on; unknown proceeds + const claudeVersion = await resolveClaudeCodeVersion(ctx.env) + + // The base URL is never written and no proxy keys appear, which is + // what keeps Remote Control's first-party predicate true with no + // override keys. + // @ref LLP 0258#settings-env [implements]: one settings write is the entire attach + // @ref LLP 0258#nothing-else [implements]: no keychain, no launchctl setenv, no LaunchAgent on this path const result = await attach({ port, version: ctx.plugin.version, stateFile, settingsPath, binPath: resolveHookBinPath(ctx.env), - ...(ca ? { mode: MODE_PROXY, caCertPath: ca.certPath } : {}), + mode: MODE_OTEL, + telemetryPort, + spoolDir, + claudeVersion, }) + // After the settings write, not before: a floor refusal must leave + // nothing behind, and Claude Code only starts writing bodies once a + // session launches with the new settings. Created owner-only here + // so raw prompts never pass through a default-mode directory. + // + // In its own try, because the settings write above has already + // landed: an unwritable spool root would otherwise report the whole + // attach as failed while the client is in fact attached, and would + // swallow the migration notes below - including the line naming + // `hyp detach claude --purge` for the CA a migrated machine still + // carries. A warning names the one thing that did not happen. + // @ref LLP 0253#spool-location [implements] + /** @type {string | undefined} */ + let spoolWarning + try { + await ensureClaudeBodySpool(spoolDir) + } catch (spoolErr) { + const detail = spoolErr instanceof Error ? spoolErr.message : String(spoolErr) + spoolWarning = + `could not prepare the raw-body spool at ${spoolDir} (${detail}); ` + + 'capture falls back to the events alone until it is writable' + logger.warn('client.attach.spool_unavailable', { + hyp_plugin: PLUGIN_NAME, + hyp_client: CLIENT_NAME, + spool_dir: spoolDir, + error: detail, + }) + } // Malformed `env` / `hooks` blocks attach rebuilt after backing the // displaced value up into the marker (LLP 0163). Reported on the // span, in the log, and to the user - the whole point of the // decision is that the repair stops being silent. - const warnings = result.changed && result.warnings !== undefined + const malformedWarnings = result.changed && result.warnings !== undefined ? [...result.warnings] : [] - - // The settings keys alone leave Remote Control's inbound channel - // broken (LLP 0236): its transport trusts only the keychain, and - // only when NODE_USE_SYSTEM_CA=1 was in the environment at boot. - // Both halves are macOS-only, both degrade to a warning rather - // than failing the attach - capture works without them. - /** @type {'granted' | 'already' | 'refused' | undefined} */ - let trustState + // The spool warning rides the same user-facing list but is counted + // apart: `malformed_blocks_repaired` names one specific repair, and + // folding an unrelated warning into it would make the count lie. + const warnings = spoolWarning === undefined + ? malformedWarnings + : [...malformedWarnings, spoolWarning] + + // A prior proxy marker makes this attach a migration. The settings + // write above already released the proxy keys (the LLP 0232 + // mode-switch rule); what is left is the residue outside the + // settings file. The launchd environment is unwound here; the CA + // trust is OFFERED, never taken: it carries the once-per-machine + // password-dialog grant, other clients may still proxy through the + // gateway, and ending the grant is `hyp detach --purge`'s job. + // @ref LLP 0262#migration [implements]: release the proxy keys, unwind the launchd env, offer detach --purge, write the OTEL block + const migratedFrom = result.changed && result.priorMode === MODE_PROXY + ? MODE_PROXY + : undefined /** @type {boolean | undefined} */ - let launchdEnvSet - if (ca && process.platform === 'darwin') { - const darwin = await ensureDarwinProxyTrust({ - certPath: ca.certPath, - hosts: ca.hosts, - stdout: attachCtx.stdout, - }) - trustState = darwin.trustState - launchdEnvSet = darwin.launchdEnvSet - warnings.push(...darwin.warnings) - span.setAttribute('proxy_trust', darwin.trustState) - span.setAttribute('launchd_env_set', darwin.launchdEnvSet) - } else if (ca) { - // @ref LLP 0237#darwin-only [implements] - warnings.push( - 'Remote Control inbound is not supported under proxy mode on this platform yet' + let launchdEnvRemoved + /** @type {string[]} */ + const migrationNotes = [] + if (migratedFrom !== undefined) { + const unwind = await unwindProxyLaunchdEnv({ homeDir }) + launchdEnvRemoved = unwind.launchdEnvRemoved + warnings.push(...unwind.warnings) + migrationNotes.push( + 'Migrated from proxy attach: the proxy env keys are released and ' + + 'Claude Code talks to Anthropic directly again.', + 'Sessions started before this keep proxying until they restart; ' + + 'the overlap dedupes into the same rows.' + ) + if (launchdEnvRemoved === true) { + migrationNotes.push('Removed NODE_USE_SYSTEM_CA from the launchd environment.') + } + // "any trust it was granted" rather than "the trust you granted": + // a proxy attach whose keychain dialog was refused still ran and + // still left the CA, so claiming a grant we never verified would + // be the one false line in the migration's story. + migrationNotes.push( + (process.platform === 'darwin' + ? 'The HypAware Local CA, and any login-keychain trust it was granted, ' + + 'is still in place. ' + : 'The HypAware local CA is still on disk. ') + + "Run 'hyp detach claude --purge' to remove it (then 'hyp attach claude' " + + 'to keep capturing); it is never removed without you asking.' ) + span.setAttribute('migrated_from', migratedFrom) + if (launchdEnvRemoved !== undefined) { + span.setAttribute('launchd_env_removed', launchdEnvRemoved) + } + logger.info('client.attach.migrated', { + hyp_plugin: PLUGIN_NAME, + hyp_client: CLIENT_NAME, + from_mode: MODE_PROXY, + to_mode: MODE_OTEL, + ...(launchdEnvRemoved !== undefined + ? { launchd_env_removed: launchdEnvRemoved } + : {}), + }) } span.setAttribute('status', 'ok') span.setAttribute('restored', false) - span.setAttribute('malformed_blocks_repaired', warnings.length) + span.setAttribute('malformed_blocks_repaired', malformedWarnings.length) logger.info('client.attach.write', { hyp_plugin: PLUGIN_NAME, hyp_client: CLIENT_NAME, settings_path: settingsPath, port, changed: result.changed === true, - malformed_blocks_repaired: warnings.length, + malformed_blocks_repaired: malformedWarnings.length, }) - for (const warning of warnings) { + for (const warning of malformedWarnings) { logger.warn('client.attach.malformed_block', { hyp_plugin: PLUGIN_NAME, hyp_client: CLIENT_NAME, @@ -270,10 +358,12 @@ export async function activate(ctx) { prevValue: result.changed && result.prevValue !== undefined ? result.prevValue : undefined, - mode: ca ? MODE_PROXY : MODE_BASE_URL, - caCertPath: ca?.certPath, - trust: trustState, - launchdEnvSet, + mode: MODE_OTEL, + telemetryPort, + spoolDir, + migratedFrom, + launchdEnvRemoved, + migrationNotes, warnings, }) } catch (err) { @@ -287,6 +377,51 @@ export async function activate(ctx) { }, }) + // The plugin's first dataset: behavioral events the wire never showed. + // Registered unconditionally (not gated on the listener below): the + // rows a past daemon wrote must stay queryable and enumerable even + // when this boot cannot host the listener. + // @ref LLP 0255#owned-by-claude [implements]: the payload shapes are Claude + // Code's, so the plugin that interprets them owns the table + ctx.query.registerDataset(claudeTelemetryDatasetRegistration()) + + // The telemetry listener: Claude Code's own OTLP export, received on + // loopback and projected into the same `ai_gateway_messages` rows the + // proxy and transcript backfill produce. Registered, not started: + // the daemon starts every registered source, so a CLI activation + // never binds a port. + // + // Feature-detected against the gateway capability so a mixed install + // (an older `@hypaware/ai-gateway` without the record seam) degrades + // to "no listener" rather than throwing at boot. + // @ref LLP 0257#registration [implements]: `@hypaware/claude` contributes the + // listener source through the kernel source registry, with its own config + // section and its own port + if (typeof gateway.recordProjectedExchange === 'function') { + ctx.sources.register({ + name: CLAUDE_TELEMETRY_SOURCE, + plugin: PLUGIN_NAME, + summary: 'Claude Code OTLP telemetry receiver: projects its event stream into ai_gateway_messages', + configSection: CLAUDE_CONFIG_SECTION, + start: createStartClaudeTelemetrySource({ + gateway, + clientName: CLIENT_NAME, + stateFile, + // The same shared-state-root list every other capture seam above gets. + // The listener is a capture seam too, and this is the arm of the policy + // that no `.hypignore` dotfile expresses. + // @ref LLP 0254#policy-inline [implements]: the machine-local list is in + // scope at ingest, not only the committable dotfile + localOnlyListPath: localOnlyList, + }), + }) + } else { + logger.warn('claude.telemetry.capability_too_old', { + hyp_plugin: PLUGIN_NAME, + detail: 'the active @hypaware/ai-gateway has no recordProjectedExchange; the telemetry listener is not registered', + }) + } + ctx.commands.register({ name: 'claude-hook session-context', summary: 'Internal Claude Code hook: appends session context to the state file', @@ -509,64 +644,53 @@ function safeEndpointPort(endpoint) { } /** - * The two macOS-only halves of a working proxy attach: user-domain keychain - * trust for the CA, and `NODE_USE_SYSTEM_CA=1` in the launchd user - * environment. Every failure is a warning, never a throw: capture works - * without either half, and the attach must say what is degraded rather than - * refuse to deliver what still works. - * @ref LLP 0237#attach-anyway-on-refusal [implements] + * Unwind the launchd-environment half of a proxy attach when `hyp attach + * claude` migrates the machine to `otel` mode. * - * The pre-dialog line is written directly: the macOS password dialog appears - * mid-attach, and a user who has not been told why gets a scary - * trust-settings prompt with no context. + * Mirrors the detach undo's release (`releaseProxyModeLaunchdEnv` in + * `client_detach_disk.js`): darwin-only, best-effort, and the same by-hand + * hint on failure, because the attach that just migrated must never fail on + * residue it can name. The CA and its keychain trust deliberately stay: they + * carry the user's once-per-machine password-dialog grant, and only `hyp + * detach --purge` or `hyp daemon uninstall` may end it. The migration OFFERS + * that step in its output; it never runs it. + * @ref LLP 0262#migration [implements]: the launchd env is unwound; the CA trust is offered, never forced + * @ref LLP 0239#launchctl-setenv [implements]: the migration is one more path that reverses a proxy attach, so it releases the env too * - * @param {{ certPath: string, hosts: string[], stdout: { write(s: string): unknown } }} args - * @returns {Promise<{ - * trustState: 'granted' | 'already' | 'refused', - * launchdEnvSet: boolean, - * warnings: string[], - * }>} + * @param {{ + * homeDir?: string, + * platform?: NodeJS.Platform, + * removeEnv?: typeof removeLaunchdEnv, + * }} [args] + * @returns {Promise<{ launchdEnvRemoved?: boolean, warnings: string[] }>} */ -async function ensureDarwinProxyTrust({ certPath, hosts, stdout }) { - /** @type {string[]} */ - const warnings = [] - /** @type {'granted' | 'already' | 'refused'} */ - let trustState - - if (await isCaTrusted({ certPath })) { - trustState = 'already' - } else { - // Name every host the trust will cover, so the grant is informed - the - // constraint set is wider than the one provider being attached. - // @ref LLP 0238#full-provider-constraints [constrained-by]: the dialog context must name all permitted hosts - stdout.write( - ` Requesting keychain trust for the HypAware Local CA (limited to: ${hosts.join(', ')}).\n` + - ' macOS will ask for your login password.\n' - ) - const install = await installCaTrust({ certPath }) - if (install.installed) { - trustState = 'granted' - } else { - trustState = 'refused' - warnings.push( - 'keychain trust was not granted' + - `${install.detail ? ` (${install.detail})` : ''}; ` + - 'capture works, but Remote Control messages sent from other devices will not arrive. ' + - 'Re-run `hyp attach claude` to retry.' - ) +export async function unwindProxyLaunchdEnv({ + homeDir, + platform = process.platform, + removeEnv = removeLaunchdEnv, +} = {}) { + if (platform !== 'darwin') return { warnings: [] } + try { + const removal = await removeEnv({ homeDir }) + if (removal.unset) return { launchdEnvRemoved: true, warnings: [] } + return { + launchdEnvRemoved: false, + warnings: [ + 'NODE_USE_SYSTEM_CA could not be unset from the launchd environment' + + `${removal.detail ? ` (${removal.detail})` : ''}; ` + + 'run `launchctl unsetenv NODE_USE_SYSTEM_CA` by hand', + ], + } + } catch (err) { + return { + launchdEnvRemoved: false, + warnings: [ + 'the launchd environment could not be released ' + + `(${err instanceof Error ? err.message : String(err)}); ` + + 'run `launchctl unsetenv NODE_USE_SYSTEM_CA` by hand', + ], } } - - const env = await installLaunchdEnv({}) - if (!env.set) { - warnings.push( - 'NODE_USE_SYSTEM_CA could not be set in the launchd environment' + - `${env.detail ? ` (${env.detail})` : ''}; ` + - 'launch Claude Code with `NODE_USE_SYSTEM_CA=1` in the shell until this is fixed.' - ) - } - - return { trustState, launchdEnvSet: env.set, warnings } } /** @@ -583,10 +707,12 @@ async function ensureDarwinProxyTrust({ certPath, hosts, stdout }) { * port: number | undefined, * changed: boolean, * prevValue?: string, - * mode?: 'proxy' | 'base_url', - * caCertPath?: string, - * trust?: 'granted' | 'already' | 'refused', - * launchdEnvSet?: boolean, + * mode?: 'proxy' | 'base_url' | 'otel', + * telemetryPort?: number, + * spoolDir?: string, + * migratedFrom?: 'proxy', + * launchdEnvRemoved?: boolean, + * migrationNotes?: string[], * warnings?: string[], * }} fields */ @@ -603,14 +729,18 @@ function writeAttachOutput(attachCtx, fields) { } if (fields.port !== undefined) payload.port = fields.port if (fields.mode !== undefined) payload.mode = fields.mode - if (fields.caCertPath !== undefined) payload.ca_cert_path = fields.caCertPath - if (fields.trust !== undefined) payload.keychain_trust = fields.trust - if (fields.launchdEnvSet !== undefined) payload.launchd_env_set = fields.launchdEnvSet + if (fields.telemetryPort !== undefined) payload.telemetry_port = fields.telemetryPort + if (fields.spoolDir !== undefined) payload.spool_dir = fields.spoolDir + // The typed migration facts, so a scripted caller can tell a migrating + // attach from a routine one without parsing prose; the human notes below + // stay off this surface. + if (fields.migratedFrom !== undefined) payload.migrated_from = fields.migratedFrom + if (fields.launchdEnvRemoved !== undefined) payload.launchd_env_removed = fields.launchdEnvRemoved // Named, because `prev_value` alone does not say which key it belonged to - // and the two modes manage different ones. + // and each mode manages different ones. if (fields.prevValue !== undefined) { payload.prev_value = fields.prevValue - payload.prev_value_key = fields.mode === MODE_PROXY ? 'HTTPS_PROXY' : 'ANTHROPIC_BASE_URL' + payload.prev_value_key = takenOverKey(fields.mode) } // Echoed as an array, not folded into a string: the field exists so a // scripted caller can see *which* blocks were moved aside. @@ -618,40 +748,58 @@ function writeAttachOutput(attachCtx, fields) { attachCtx.stdout.write(JSON.stringify(payload) + '\n') return } - // Name the key actually written. Proxy mode does not set a base URL at all, - // and reporting one is both wrong and the first thing a user would check when - // debugging why their own base URL is still in place. - const managedKey = fields.mode === MODE_PROXY ? 'HTTPS_PROXY' : 'ANTHROPIC_BASE_URL' + // Name the key actually written. Proxy and otel modes do not set a base URL + // at all, and reporting one is both wrong and the first thing a user would + // check when debugging why their own base URL is still in place. + const managedKey = takenOverKey(fields.mode) if (fields.dryRun) { attachCtx.stdout.write(`(dry-run) Would attach Claude Code via ${fields.settingsPath}\n`) - attachCtx.stdout.write(` Would set ${managedKey} to the local gateway endpoint\n`) + attachCtx.stdout.write( + fields.mode === MODE_OTEL + ? ` Would set ${managedKey} to the local telemetry listener\n` + : ` Would set ${managedKey} to the local gateway endpoint\n` + ) return } attachCtx.stdout.write(`✓ Claude Code attached (${fields.settingsPath})\n`) - if (fields.port !== undefined) { + if (fields.mode === MODE_OTEL) { + // The two values a user would check: where the events go, and where raw + // bodies land until the listener projects and deletes them. + if (fields.telemetryPort !== undefined) { + attachCtx.stdout.write(` ${managedKey} = http://127.0.0.1:${fields.telemetryPort}\n`) + } + if (fields.spoolDir !== undefined) { + attachCtx.stdout.write(` OTEL_LOG_RAW_API_BODIES = file:${fields.spoolDir}\n`) + } + } else if (fields.port !== undefined) { attachCtx.stdout.write(` ${managedKey} = http://127.0.0.1:${fields.port}\n`) } - if (fields.mode === MODE_PROXY && fields.caCertPath !== undefined) { - attachCtx.stdout.write(` NODE_EXTRA_CA_CERTS = ${fields.caCertPath}\n`) - } if (fields.prevValue !== undefined) { attachCtx.stdout.write(` (previous ${managedKey} was ${fields.prevValue})\n`) } + // The migration story, told where the user is looking: what the switch + // released, what was unwound, and the one residue that is theirs to end + // (the CA trust, offered as `hyp detach claude --purge`, never run for + // them). + // @ref LLP 0262#migration [implements]: the offer is a printed step, not an action + for (const note of fields.migrationNotes ?? []) { + attachCtx.stdout.write(` ${note}\n`) + } for (const warning of fields.warnings ?? []) { attachCtx.stdout.write(` ! ${warning}\n`) } - // Last, so it is the line the user acts on. `launchctl setenv` reaches only - // processes launchd starts afterwards; windows of an already-running - // terminal app inherit the app's stale environment, so "open a new window" - // is not enough (proven in the run G acceptance test). - // @ref LLP 0239#terminals-predating-attach [implements]: already-open terminal apps are told to relaunch, not fixed - if (fields.mode === MODE_PROXY && fields.launchdEnvSet === true && fields.trust !== 'refused') { - attachCtx.stdout.write( - ' One more step for Remote Control: quit your terminal app completely ' + - '(Cmd-Q) and reopen it.\n' + - ' A new window or tab is not enough; apps launched from now on pick up ' + - 'the change automatically.\n' - ) - } +} + +/** + * The env key each attach mode takes over: the one a displaced `prev_value` + * belonged to, and the one the human output leads with. + * + * @param {'proxy' | 'base_url' | 'otel' | undefined} mode + * @returns {'HTTPS_PROXY' | 'OTEL_EXPORTER_OTLP_ENDPOINT' | 'ANTHROPIC_BASE_URL'} + */ +function takenOverKey(mode) { + if (mode === MODE_PROXY) return 'HTTPS_PROXY' + if (mode === MODE_OTEL) return 'OTEL_EXPORTER_OTLP_ENDPOINT' + return 'ANTHROPIC_BASE_URL' } diff --git a/hypaware-core/plugins-workspace/claude/src/settings.js b/hypaware-core/plugins-workspace/claude/src/settings.js index ba3d0beb..331836cc 100644 --- a/hypaware-core/plugins-workspace/claude/src/settings.js +++ b/hypaware-core/plugins-workspace/claude/src/settings.js @@ -12,6 +12,7 @@ import { redactUrlUserinfo, } from 'hypaware/core/util' import { markActionRefused } from '../../../../src/core/config/action_refusal.js' +import { CLAUDE_OTEL_MIN_VERSION, CLAUDE_UPDATE_HINT, isBelowClaudeVersion } from './claude_version.js' /** * Claude Code settings.json attach writer, keyed on the `_hypaware` @@ -36,6 +37,13 @@ import { markActionRefused } from '../../../../src/core/config/action_refusal.js * rebuilt before attach could write into it. Attach repairs rather than * refuses, and the marker is what makes the repair reversible and * reportable instead of destructive. See LLP 0163. + * + * Three modes share all of that machinery unchanged. `base_url` repoints + * `ANTHROPIC_BASE_URL` at the gateway, `proxy` sets `HTTPS_PROXY` plus a + * CA, and `otel` turns on Claude Code's own telemetry export and routes + * no traffic at all. Switching between them is the same key release in + * every direction (see `releaseUnmanagedKeys`), so the marker stays the + * whole undo record whichever mode wrote it. */ /** @@ -108,6 +116,96 @@ const PROXY_MODE_ENV_KEYS = ['HTTPS_PROXY', 'NODE_EXTRA_CA_CERTS'] export const MODE_PROXY = 'proxy' /** @type {'base_url'} */ export const MODE_BASE_URL = 'base_url' +/** @type {'otel'} */ +export const MODE_OTEL = 'otel' + +/** + * The env block `otel` mode writes, in order. + * + * The list *is* the decision, which is why it is spelled out here rather than + * assembled from flags: it is the exported contract between attach, the + * listener that receives what these flags turn on, and the spool sweep. Note + * what is absent - no `ANTHROPIC_BASE_URL`, no `HTTPS_PROXY`, no + * `NODE_EXTRA_CA_CERTS` - which is what leaves the endpoint first-party and + * Remote Control working with no override keys at all. + * + * Unlike the base-URL mode's additions these are take-over keys, handled like + * the proxy keys: a user who already points Claude Code at their own collector + * has that value backed up into `prev_env` and restored on detach, rather than + * being skipped (which would leave attach reporting success while the events + * went somewhere else). + * + * @ref LLP 0258#env-keys [implements]: exactly these keys, and only these + * @param {{ telemetryPort: number, spoolDir: string }} args + * @returns {{ key: string, value: string }[]} + */ +export function otelModeEnv({ telemetryPort, spoolDir }) { + return [ + { key: 'CLAUDE_CODE_ENABLE_TELEMETRY', value: '1' }, + { key: 'OTEL_LOGS_EXPORTER', value: 'otlp' }, + { key: 'OTEL_METRICS_EXPORTER', value: 'otlp' }, + { key: 'OTEL_EXPORTER_OTLP_PROTOCOL', value: 'http/json' }, + { key: 'OTEL_EXPORTER_OTLP_ENDPOINT', value: `http://127.0.0.1:${telemetryPort}` }, + { key: 'OTEL_LOG_USER_PROMPTS', value: '1' }, + { key: 'OTEL_LOG_ASSISTANT_RESPONSES', value: '1' }, + { key: 'OTEL_LOG_TOOL_DETAILS', value: '1' }, + { key: 'OTEL_LOG_RAW_API_BODIES', value: `file:${spoolDir}` }, + ] +} + +/** + * The OTLP env keys that OUTRANK the ones {@link otelModeEnv} writes. + * + * In the OTLP environment-variable contract a per-signal key beats the generic + * one, so `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` decides where log records go no + * matter what `OTEL_EXPORTER_OTLP_ENDPOINT` says. Attach deliberately does not + * manage these (LLP 0258 #env-keys is "exactly these keys, and only these"), + * which leaves one shape that has to be said out loud rather than discovered: + * a user already exporting to their own collector through a per-signal key + * gets `OTEL_LOG_USER_PROMPTS` and `OTEL_LOG_ASSISTANT_RESPONSES` turned on by + * this attach, and their prompts and assistant responses start flowing THERE, + * while HypAware reports `attached (otel)` and captures nothing. + * + * `OTEL_EXPORTER_OTLP_HEADERS` is in the list for the same reason from the + * other side: it is the key that carries a collector's credentials, and it + * would now ride requests aimed at our listener. + */ +const OTEL_PER_SIGNAL_OVERRIDE_KEYS = [ + 'OTEL_EXPORTER_OTLP_LOGS_ENDPOINT', + 'OTEL_EXPORTER_OTLP_METRICS_ENDPOINT', + 'OTEL_EXPORTER_OTLP_LOGS_PROTOCOL', + 'OTEL_EXPORTER_OTLP_METRICS_PROTOCOL', + 'OTEL_EXPORTER_OTLP_HEADERS', +] + +/** + * Warn for each per-signal OTLP key left standing in the settings `env` block + * after an `otel` attach. + * + * A warning, not a refusal: attach has no way to see a key exported from the + * user's shell, so refusing on the half it CAN see would buy a false sense of + * completeness. The values are never echoed - an endpoint or a headers value + * is exactly where a collector token lives, and this string is printed, logged + * and serialised into `--json`. + * + * @param {Record} env the live `env` block, after the write + * @returns {string[]} + */ +function perSignalOverrideWarnings(env) { + /** @type {string[]} */ + const out = [] + for (const key of OTEL_PER_SIGNAL_OVERRIDE_KEYS) { + const value = env[key] + if (value === undefined || value === null || value === '') continue + out.push( + `env.${key} is set and outranks the endpoint hypaware just wrote; ` + + 'Claude Code will export there instead, including the prompt and response ' + + 'text this attach turns on. Remove it, or point it at the same local ' + + 'listener, then re-run hyp attach claude' + ) + } + return out +} export class ClaudeSettingsError extends Error { /** @@ -153,13 +251,35 @@ export async function attach(opts) { binPath = 'hyp', mode = MODE_BASE_URL, caCertPath, + telemetryPort, + spoolDir, + claudeVersion, } = opts validatePort(port) validateVersion(version) validateStateFile(stateFile) - if (mode !== MODE_PROXY && mode !== MODE_BASE_URL) { + if (mode !== MODE_PROXY && mode !== MODE_BASE_URL && mode !== MODE_OTEL) { throw new ClaudeSettingsError(`unknown attach mode: ${String(mode)}`, { code: 'INVALID_MODE' }) } + if (mode === MODE_OTEL) { + // Refused *before the settings file is even read*, which is the whole + // content of "leaves any existing attach untouched": a machine on the old + // client keeps whatever attach it already had, rather than being moved to + // a mode that captures nothing. There is deliberately no fallback to proxy + // or base-URL mode here - one attach mode per client - so the refusal is + // an error the caller reports, not a quiet downgrade. + // @ref LLP 0258#version-floor [implements]: below the floor attach refuses the switch and prints the upgrade hint + if (isBelowClaudeVersion(claudeVersion, CLAUDE_OTEL_MIN_VERSION)) { + throw markActionRefused(new ClaudeSettingsError( + `Claude Code ${String(claudeVersion)} is older than ${CLAUDE_OTEL_MIN_VERSION}, ` + + 'which is the first release that exports the telemetry HypAware captures; ' + + `run '${CLAUDE_UPDATE_HINT}' and attach again`, + { code: 'VERSION_FLOOR' } + )) + } + validateTelemetryPort(telemetryPort) + validateSpoolDir(spoolDir) + } // Proxy mode routes *all* of Claude Code's HTTPS through the gateway, so an // attach that lands without a working local CA does not degrade to // unrecorded-but-working: it breaks authentication, updates and model calls @@ -192,6 +312,22 @@ export async function attach(opts) { const { value, mtimeMs } = await readSettings(settingsPath) const priorMarker = isPlainObject(value[MARKER_KEY]) ? value[MARKER_KEY] : undefined + // What the marker said before this run rewrites it. A proxy attach leaves + // residue no settings write reaches (the launchd environment, the keychain + // trust), and by the time the caller could re-read the marker this write has + // already replaced it, so the prior mode is reported on the result. Only the + // three known modes are reported: a legacy marker without one predates modes + // entirely and has no residue to unwind. + // @ref LLP 0262#migration [implements]: the prior mode is what tells the adapter a proxy attach is being migrated + /** @type {'proxy' | 'base_url' | 'otel' | undefined} */ + let priorMode + if ( + priorMarker && + (priorMarker.mode === MODE_PROXY || priorMarker.mode === MODE_BASE_URL || priorMarker.mode === MODE_OTEL) + ) { + priorMode = priorMarker.mode + } + // A backup an earlier run already recorded at some path. Read before anything // is displaced, because it decides what this run is allowed to claim: a prior // entry wins (see below), so a value displaced *this* run at an @@ -373,6 +509,35 @@ export async function attach(opts) { managedEnv.HTTPS_PROXY = `http://127.0.0.1:${port}` managedEnv.NODE_EXTRA_CA_CERTS = /** @type {string} */ (caCertPath) for (const [key, next] of Object.entries(managedEnv)) env[key] = next + } else if (mode === MODE_OTEL) { + // Claude Code talks to Anthropic directly and exports its own telemetry to + // us, so nothing here routes traffic: the endpoint stays first party and + // the Remote Control predicate holds without a single override key. + // @ref LLP 0258#env-keys [implements] + // @ref LLP 0258#settings-env [implements]: the settings `env` block is the only surface attach writes + const additions = otelModeEnv({ + telemetryPort: /** @type {number} */ (telemetryPort), + spoolDir: /** @type {string} */ (spoolDir), + }) + for (const { key } of additions) { + const prior = priorValueFor(key) + if (prior.value !== undefined) prevEnv[key] = prior.value + if (prior.carriedForward || prior.value === undefined) continue + // A pre-existing OTEL key is almost always a user's own collector, and + // taking it over silently would send their telemetry here instead. The + // value is backed up and restored on detach either way, but only the run + // that displaced it has anything new to say. The value itself is not + // echoed: an endpoint or a headers value is exactly where a collector + // token ends up, and this string is printed and logged. + warnings.push( + `env.${key} was already set; hypaware now manages it and hyp detach restores it` + ) + } + for (const { key, value } of additions) { + managedEnv[key] = value + env[key] = value + } + warnings.push(...perSignalOverrideWarnings(env)) } else { // Undo the defaults Claude Code flips because the gateway URL is not // api.anthropic.com: eager tool-schema loading, and a 200k assumed context @@ -432,6 +597,12 @@ export async function attach(opts) { ...(mode === MODE_BASE_URL && prevBaseUrl !== undefined ? { prev_base_url: prevBaseUrl } : {}), + // The one thing about an `otel` attach that is not derivable from the + // managed keys: detach and `hyp purge` have to empty a directory neither + // of them computed, and the env value that names it is gone by the time + // they run. + // @ref LLP 0258#marker-and-spool [implements]: the marker records the spool directory + ...(mode === MODE_OTEL ? { spool_dir: spoolDir } : {}), ...(Object.keys(prevEnv).length > 0 ? { prev_env: prevEnv } : {}), ...(Object.keys(prevMalformed).length > 0 ? { prev_malformed: prevMalformed } : {}), } @@ -440,16 +611,26 @@ export async function attach(opts) { /** @type {ClaudeAttachResult} */ const result = { changed: true } - const reportedPrev = mode === MODE_PROXY ? prevEnv.HTTPS_PROXY : prevBaseUrl + if (priorMode !== undefined) result.priorMode = priorMode + // Each mode reports the key it actually took over. Reporting a displaced + // base URL from a mode that never touched `ANTHROPIC_BASE_URL` would be the + // first thing a user checked when their own value turned out to still be + // there. + const reportedPrev = mode === MODE_PROXY + ? prevEnv.HTTPS_PROXY + : mode === MODE_OTEL + ? prevEnv.OTEL_EXPORTER_OTLP_ENDPOINT + : prevBaseUrl if (reportedPrev !== undefined) { const shown = typeof reportedPrev === 'string' ? reportedPrev : String(reportedPrev) // A display field, not the backup: the marker above already holds the true // value, and this one is printed and serialised into `prev_value`. In proxy - // mode it is a `HTTPS_PROXY` that routinely carries `user:pass@`, so the - // userinfo comes off the copy the user and any `--json` consumer see. Base - // URLs go through unchanged: `ANTHROPIC_BASE_URL` carries no userinfo, and - // the value is the whole point of the notice. - result.prevValue = mode === MODE_PROXY ? redactUrlUserinfo(shown) : shown + // mode it is a `HTTPS_PROXY` that routinely carries `user:pass@`, and in + // `otel` mode a collector endpoint that can carry the same, so the userinfo + // comes off the copy the user and any `--json` consumer see. Base URLs go + // through unchanged: `ANTHROPIC_BASE_URL` carries no userinfo, and the + // value is the whole point of the notice. + result.prevValue = mode === MODE_BASE_URL ? shown : redactUrlUserinfo(shown) } // Only what *this* run displaced. A re-attach carries the prior backup on the // marker but has nothing new to tell the user about, so it warns about @@ -790,6 +971,51 @@ function validateVersion(version) { } } +/** + * The listener port `otel` mode points Claude Code at. A separate validator + * from {@link validatePort} so the error names the option the caller passed: + * two ports reach `attach()` in this mode, and "invalid port" would not say + * which one. + * + * @param {unknown} telemetryPort + */ +function validateTelemetryPort(telemetryPort) { + if ( + typeof telemetryPort !== 'number' || + !Number.isInteger(telemetryPort) || + telemetryPort < 1 || + telemetryPort > 65535 + ) { + throw new ClaudeSettingsError( + `otel-mode attach requires the telemetry listener port, got '${String(telemetryPort)}'`, + { code: 'INVALID_TELEMETRY_PORT' } + ) + } +} + +/** + * Absolute, because the value goes into `OTEL_LOG_RAW_API_BODIES` and Claude + * Code resolves it against *its own* working directory. A relative path there + * would scatter raw request bodies through every repo the user works in, + * outside the HypAware home that `hyp purge` and detach sweep. + * + * @ref LLP 0253#spool-location [constrained-by]: the spool lives under the HypAware home + * @param {unknown} spoolDir + */ +function validateSpoolDir(spoolDir) { + if (typeof spoolDir !== 'string' || spoolDir.length === 0) { + throw new ClaudeSettingsError('otel-mode attach requires the body spool directory', { + code: 'INVALID_SPOOL_DIR', + }) + } + if (!path.isAbsolute(spoolDir)) { + throw new ClaudeSettingsError( + `spoolDir must be an absolute path, got '${spoolDir}'`, + { code: 'INVALID_SPOOL_DIR' } + ) + } +} + /** @param {unknown} stateFile */ function validateStateFile(stateFile) { if (typeof stateFile !== 'string' || stateFile.length === 0) { diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/bodies.js b/hypaware-core/plugins-workspace/claude/src/telemetry/bodies.js new file mode 100644 index 00000000..6f43d474 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/bodies.js @@ -0,0 +1,363 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' + +import { isPlainObject, parseMaybeJson, stringValue } from 'hypaware/core/util' +import { anthropicConversationFields, anthropicMessageAttributes } from '../anthropic.js' + +/** + * @import { AiGatewayProjectedMessage, JsonObject } from '../../../../../hypaware-plugin-kernel-types.js' + * @import { ClaudeTelemetryEvent, SpooledClaudeBody } from '../types.js' + */ + +/** + * The two event names that reference a spooled body file. Everything + * about the file's location comes from the event's `body_ref` + * attribute: Claude Code writes the absolute path of the file it just + * dropped into the spool. + */ +export const BODY_EVENT_NAMES = Object.freeze(['api_request_body', 'api_response_body']) + +/** + * Content-block types a body file is consulted for. Text blocks are + * deliberately absent: the `user_prompt` and `assistant_response` + * events already delivered that content once, keyed by native uuid, + * and re-projecting it from the body would store every text twice + * under a second identity. + * + * @ref LLP 0252#bodies-for-gaps [implements]: a body is read for what events + * lack (untruncated tool args, thinking signatures, tool results, ordering) + * and for nothing else + */ +const GAP_BLOCK_TYPES = new Set([ + 'tool_use', + 'server_tool_use', + 'thinking', + 'redacted_thinking', + 'tool_result', + 'web_search_tool_result', +]) + +/** + * Read the spooled body files a batch of events references. + * + * Only files inside the spool directory are touched: `body_ref` arrives + * over the wire from whatever process found the loopback port, and this + * listener both reads and DELETES what it names, so an uncontained ref + * would turn the event stream into a read-and-delete primitive over the + * whole filesystem. Refs outside the spool are refused, counted, and + * left alone. + * + * A file that fails to parse is deleted immediately and counted: the + * same session is recoverable from transcript backfill, and an + * undeleted body is a raw prompt sitting on disk. + * @ref LLP 0252#project-then-delete [implements]: an unprojectable body is + * deleted and counted, not retried forever + * + * @param {ClaudeTelemetryEvent[]} events + * @param {{ spoolDir: string }} opts + * @returns {Promise<{ + * bodies: Map, + * consumedFiles: string[], + * consumedBytes: number, + * missing: number, + * unparseable: number, + * refused: string[], + * }>} + */ +export async function loadSpooledBodies(events, opts) { + /** @type {Map} */ + const bodies = new Map() + /** @type {string[]} */ + const consumedFiles = [] + let consumedBytes = 0 + let missing = 0 + let unparseable = 0 + /** @type {string[]} */ + const refused = [] + + const spoolRoot = path.resolve(opts.spoolDir) + for (const event of events) { + if (!BODY_EVENT_NAMES.includes(event.name)) continue + const ref = stringValue(event.attributes.body_ref) + if (!ref || bodies.has(ref)) continue + const file = path.resolve(ref) + if (!file.startsWith(spoolRoot + path.sep)) { + refused.push(ref) + continue + } + /** @type {Buffer} */ + let raw + try { + raw = await fs.readFile(file) + } catch { + // Already projected, already evicted, or never written: the + // content is recoverable from the transcript either way. + // @ref LLP 0253#eviction-degrades [implements]: an evicted body is not an + // error; the failure mode is "captured later, with less detail" + missing += 1 + continue + } + const body = parseMaybeJson(raw.toString('utf8')) + if (!isPlainObject(body)) { + unparseable += 1 + await fs.rm(file, { force: true }).catch(() => {}) + continue + } + bodies.set(ref, { + kind: event.name === 'api_request_body' ? 'request' : 'response', + file, + body, + }) + consumedFiles.push(file) + consumedBytes += raw.length + } + + return { bodies, consumedFiles, consumedBytes, missing, unparseable, refused } +} + +/** + * Delete body files after their content has been projected and written. + * + * Called only after the batch's dataset writes succeeded: a write + * failure surfaces as an HTTP error the exporter retries, and the + * retried batch re-reads the same files. Deletion is the normal end of + * a body's life, which is what keeps the spool transient. + * + * @ref LLP 0252#project-then-delete [implements]: a body file is deleted as + * soon as it has been projected + * @param {string[]} files + * @returns {Promise} how many files were removed + */ +export async function deleteSpooledBodies(files) { + let deleted = 0 + for (const file of files) { + try { + await fs.rm(file, { force: true }) + deleted += 1 + } catch { + // A vanished or unremovable file is the sweep's problem, not a + // reason to fail the batch that already recorded its content. + } + } + return deleted +} + +/** + * Delete the spooled bodies a set of events references WITHOUT reading + * them: the deletion arm of a policy drop. When ingest drops a session + * (a per-session ignore today; the usage-policy governors take the same + * path), its body files must not sit in the spool until the cap evicts + * them - the content of exactly the session the user asked us not to + * keep. The events' `body_ref`s are resolved under the same + * spool-containment rule as `loadSpooledBodies`, refs outside the spool + * are refused and counted, and nothing is parsed or projected. + * + * @ref LLP 0253#delete-on-drop [implements]: a dropped session's bodies are + * deleted, never merely skipped + * @ref LLP 0256#bodies-deleted [implements]: the session-ignore transport + * works AND the content goes + * @param {ClaudeTelemetryEvent[]} events + * @param {{ spoolDir: string }} opts + * @returns {Promise<{ deleted: number, refused: string[] }>} + */ +export async function deleteSpooledBodiesForEvents(events, opts) { + const spoolRoot = path.resolve(opts.spoolDir) + /** @type {string[]} */ + const files = [] + /** @type {string[]} */ + const refused = [] + /** @type {Set} */ + const seen = new Set() + for (const event of events) { + if (!BODY_EVENT_NAMES.includes(event.name)) continue + const ref = stringValue(event.attributes.body_ref) + if (!ref || seen.has(ref)) continue + seen.add(ref) + const file = path.resolve(ref) + if (!file.startsWith(spoolRoot + path.sep)) { + refused.push(ref) + continue + } + files.push(file) + } + const deleted = await deleteSpooledBodies(files) + return { deleted, refused } +} + +/** + * The exchange-level fields a request body supplies: the system prompt, + * the tool declarations, and the model. These are columns stamped on + * every row of the projection, which events never carry. + * + * @param {SpooledClaudeBody} spooled + * @returns {{ system_text?: string, tools?: unknown, model?: string }} + */ +export function requestBodyFacts(spooled) { + if (spooled.kind !== 'request') return {} + const fields = anthropicConversationFields(spooled.body, undefined) + /** @type {{ system_text?: string, tools?: unknown, model?: string }} */ + const facts = {} + if (fields.system_text) facts.system_text = fields.system_text + if (fields.tools !== undefined) facts.tools = fields.tools + if (fields.model) facts.model = fields.model + return facts +} + +/** + * Project one spooled body into the messages events cannot supply. + * + * A request body contributes its message history's gap blocks in + * canonical order (tool results above all: they never appear on the + * wire as events with content). A response body contributes the + * assistant's tool_use and thinking blocks: full untruncated `input` + * where the event's `tool_input` clips at 512 characters, and the + * thinking signature the events do not carry at all. + * + * Each block becomes its own projected message, mirroring the proxy + * path's per-block decomposition, so the gateway's fallback content + * hash gives the same block the same identity from either producer and + * the repeated history of the next turn's request dedupes away. + * + * A response with no text block never produces an `assistant_response` + * event, so its usage would otherwise go unclaimed: the last gap block + * carries it (from the `api_request` event when one arrived, else from + * the body's own `usage`), along with the body's `stop_reason`. A + * response WITH a text block leaves usage to the event that carries the + * text, so a SUM over rows never counts a request twice. + * + * @param {SpooledClaudeBody} spooled + * @param {{ + * event: ClaudeTelemetryEvent, + * usageByRequestId: Map>, + * }} ctx + * @returns {AiGatewayProjectedMessage[]} + */ +export function spooledBodyGapMessages(spooled, ctx) { + return spooled.kind === 'request' + ? requestGapMessages(spooled, ctx.event) + : responseGapMessages(spooled, ctx) +} + +/** + * @param {SpooledClaudeBody} spooled + * @param {ClaudeTelemetryEvent} event + * @returns {AiGatewayProjectedMessage[]} + */ +function requestGapMessages(spooled, event) { + const messages = Array.isArray(spooled.body.messages) ? spooled.body.messages : [] + const frame = bodyFrame(spooled, event) + /** @type {AiGatewayProjectedMessage[]} */ + const out = [] + for (const message of messages) { + if (!isPlainObject(message)) continue + const role = stringValue(message.role) + if (!role) continue + for (const block of gapBlocks(message.content)) { + out.push(gapMessage({ role, block, event, frame })) + } + } + return out +} + +/** + * @param {SpooledClaudeBody} spooled + * @param {{ event: ClaudeTelemetryEvent, usageByRequestId: Map> }} ctx + * @returns {AiGatewayProjectedMessage[]} + */ +function responseGapMessages(spooled, ctx) { + const { event } = ctx + const body = spooled.body + if (stringValue(body.role) !== 'assistant') return [] + const content = Array.isArray(body.content) ? body.content : [] + const kept = gapBlocks(content) + if (kept.length === 0) return [] + + const frame = bodyFrame(spooled, event) + const requestId = stringValue(event.attributes.request_id) + const model = stringValue(body.model) + const hasText = content.some((block) => isPlainObject(block) && block.type === 'text') + + /** @type {AiGatewayProjectedMessage[]} */ + const out = [] + for (let i = 0; i < kept.length; i++) { + const message = gapMessage({ role: 'assistant', block: kept[i], event, frame }) + if (requestId) message.request_id = requestId + if (model) message.model = model + if (i === kept.length - 1 && !hasText) { + const stopReason = stringValue(body.stop_reason) + if (stopReason) message.stop_reason = stopReason + const usage = (requestId ? claimUsage(ctx.usageByRequestId, requestId) : undefined) + ?? anthropicMessageAttributes(body) + if (usage) message.attributes = /** @type {any} */ (usage) + } + out.push(message) + } + return out +} + +/** + * @param {{ + * role: string, + * block: Record, + * event: ClaudeTelemetryEvent, + * frame: JsonObject, + * }} args + * @returns {AiGatewayProjectedMessage} + */ +function gapMessage({ role, block, event, frame }) { + /** @type {AiGatewayProjectedMessage} */ + const message = { role, content: /** @type {any} */ ([block]), raw_frame: frame } + if (event.timestamp) message.message_created_at = event.timestamp + const promptId = stringValue(event.attributes['prompt.id']) + if (promptId) message.prompt_id = promptId + return message +} + +/** + * @param {unknown} content + * @returns {Array>} + */ +function gapBlocks(content) { + if (!Array.isArray(content)) return [] + return content.filter( + (block) => isPlainObject(block) && + typeof block.type === 'string' && + GAP_BLOCK_TYPES.has(block.type) + ) +} + +/** + * The minimized frame a body-derived row carries: enough to trace the + * row back to the body file and API exchange it came from, never any + * content. Same policy as the proxy path's minimized transcript frame. + * + * @param {SpooledClaudeBody} spooled + * @param {ClaudeTelemetryEvent} event + * @returns {JsonObject} + */ +function bodyFrame(spooled, event) { + /** @type {JsonObject} */ + const frame = { + type: spooled.kind === 'request' ? 'api_request_body' : 'api_response_body', + body_file: path.basename(spooled.file), + } + const responseId = spooled.kind === 'response' ? stringValue(spooled.body.id) : undefined + if (responseId) frame.message_id = responseId + const requestId = stringValue(event.attributes.request_id) + if (requestId) frame.request_id = requestId + if (event.timestamp) frame.timestamp = event.timestamp + return frame +} + +/** + * @param {Map>} index + * @param {string} requestId + * @returns {Record | undefined} + */ +function claimUsage(index, requestId) { + const usage = index.get(requestId) + if (usage) index.delete(requestId) + return usage +} diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/events.js b/hypaware-core/plugins-workspace/claude/src/telemetry/events.js new file mode 100644 index 00000000..3edad744 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/events.js @@ -0,0 +1,284 @@ +// @ts-check + +/** + * @import { ClaudeTelemetryEvent } from '../types.js' + */ + +/** + * Instrumentation scope Claude Code stamps on its telemetry log + * records (`com.anthropic.claude_code.events`). Everything this + * listener understands comes from that scope; anything else on the + * wire is another exporter that found the port and is ignored rather + * than half-parsed. + */ +export const CLAUDE_EVENT_SCOPE_PREFIX = 'com.anthropic.claude_code' + +/** + * Resource attribute the daemon stamps on its OWN telemetry + * (`src/core/observability/resource.js`). A daemon exporting into a + * listener the daemon hosts is the loop LLP 0021 forbids. + */ +const SELF_MARKER_KEY = 'hypaware.self' + +/** + * Decode one OTLP/JSON logs envelope into flat Claude Code events. + * + * The transport (routing, content type, encoding, response envelope) is + * the shared core server's; this is the payload half, and it is + * deliberately Claude-shaped: the record's identity is the `event.name` + * attribute, the timestamp is the `event.timestamp` attribute Claude + * Code sends as ISO-8601, and the OTLP `AnyValue` wrappers are unwrapped + * so the projector never sees `{ stringValue: ... }`. + * + * Never throws on a malformed envelope: a missing array, a null record, + * or an attribute with no recognizable value type simply contributes + * nothing. An exporter we cannot fix from our side must not be able to + * fail the request. + * + * @ref LLP 0257#registration [implements]: the shared server carries the + * transport; payload interpretation is claude-owned, including the + * self-telemetry loop guard + * @param {unknown} data OTLP/JSON `ExportLogsServiceRequest` + * @returns {ClaudeTelemetryEvent[]} + */ +export function flattenClaudeTelemetryEvents(data) { + /** @type {ClaudeTelemetryEvent[]} */ + const events = [] + const root = asObject(data) + if (!root) return events + const groups = Array.isArray(root.resourceLogs) ? root.resourceLogs : [] + + for (const groupValue of groups) { + const group = asObject(groupValue) + if (!group) continue + if (resourceHasSelfMarker(group.resource)) continue + const scopes = Array.isArray(group.scopeLogs) ? group.scopeLogs : [] + for (const scopeValue of scopes) { + const scopeLog = asObject(scopeValue) + if (!scopeLog) continue + const scopeName = stringOf(asObject(scopeLog.scope)?.name) + if (!scopeName || !scopeName.startsWith(CLAUDE_EVENT_SCOPE_PREFIX)) continue + const records = Array.isArray(scopeLog.logRecords) ? scopeLog.logRecords : [] + for (const recordValue of records) { + const event = eventFromRecord(recordValue) + if (event) events.push(event) + } + } + } + + return events +} + +/** + * Decode one OTLP/JSON metrics envelope into the same flat event shape. + * + * Claude Code exports its activity counters (`claude_code.cost.usage`, + * `claude_code.lines_of_code.count`, `claude_code.active_time.total`, + * ...) as OTLP metrics on the same exporter config, and they are part of + * the behavioral record LLP 0255 gives a home: one data point becomes + * one event, named by the metric, its data-point attributes joined by + * `value` (and `unit` when the metric declares one). The same scope and + * self-marker guards apply, and the same never-throw posture: a shape + * this decoder does not recognize contributes nothing. + * + * @ref LLP 0255#row-shape [implements]: a metric data point is an event too - + * one row, named by the metric, value and attributes preserved + * @param {unknown} data OTLP/JSON `ExportMetricsServiceRequest` + * @returns {ClaudeTelemetryEvent[]} + */ +export function flattenClaudeTelemetryMetrics(data) { + /** @type {ClaudeTelemetryEvent[]} */ + const events = [] + const root = asObject(data) + if (!root) return events + const groups = Array.isArray(root.resourceMetrics) ? root.resourceMetrics : [] + + for (const groupValue of groups) { + const group = asObject(groupValue) + if (!group) continue + if (resourceHasSelfMarker(group.resource)) continue + const scopes = Array.isArray(group.scopeMetrics) ? group.scopeMetrics : [] + for (const scopeValue of scopes) { + const scopeMetric = asObject(scopeValue) + if (!scopeMetric) continue + const scopeName = stringOf(asObject(scopeMetric.scope)?.name) + if (!scopeName || !scopeName.startsWith(CLAUDE_EVENT_SCOPE_PREFIX)) continue + const metrics = Array.isArray(scopeMetric.metrics) ? scopeMetric.metrics : [] + for (const metricValue of metrics) { + const metric = asObject(metricValue) + if (!metric) continue + const name = stringOf(metric.name) + if (!name) continue + const unit = stringOf(metric.unit) + for (const pointValue of metricDataPoints(metric)) { + const event = eventFromDataPoint(name, unit, pointValue) + if (event) events.push(event) + } + } + } + } + + return events +} + +/** + * @param {string} name + * @param {string | undefined} unit + * @param {unknown} value one OTLP `NumberDataPoint` + * @returns {ClaudeTelemetryEvent | undefined} + */ +function eventFromDataPoint(name, unit, value) { + const point = asObject(value) + if (!point) return undefined + const attributes = decodeAttributes(point.attributes) + // `value`/`unit` are plain keys (not `metric.`-namespaced) so SQL + // reads them as `JSON_VALUE(attributes, '$.value')`; no Claude Code + // data point carries attributes by those names. + const pointValue = numberOf(point.asDouble) ?? numberOf(point.asInt) ?? numberOf(point.sum) + if (pointValue !== undefined) attributes.value = pointValue + if (unit) attributes.unit = unit + const timestamp = isoFromUnixNano(point.timeUnixNano) ?? isoFromUnixNano(point.startTimeUnixNano) + return { + name, + attributes, + ...(timestamp ? { timestamp } : {}), + } +} + +/** + * The data points under whichever aggregation the metric carries. + * Claude Code's are all sums (counters); gauge and histogram are read + * too so an upstream change of aggregation degrades to "value read + * differently", not to a dropped metric. + * + * @param {Record} metric + * @returns {unknown[]} + */ +function metricDataPoints(metric) { + for (const key of ['sum', 'gauge', 'histogram']) { + const aggregation = asObject(metric[key]) + if (aggregation && Array.isArray(aggregation.dataPoints)) return aggregation.dataPoints + } + return [] +} + +/** + * @param {unknown} value + * @returns {ClaudeTelemetryEvent | undefined} + */ +function eventFromRecord(value) { + const record = asObject(value) + if (!record) return undefined + const attributes = decodeAttributes(record.attributes) + const name = stringOf(attributes['event.name']) + if (!name) return undefined + const timestamp = stringOf(attributes['event.timestamp']) + ?? isoFromUnixNano(record.timeUnixNano) + ?? isoFromUnixNano(record.observedTimeUnixNano) + const sequence = numberOf(attributes['event.sequence']) + return { + name, + attributes, + ...(timestamp ? { timestamp } : {}), + ...(sequence !== undefined ? { sequence } : {}), + } +} + +/** + * Unwrap an OTLP `KeyValue[]` into a plain object. Values keep their + * natural JS type where OTLP names one (`intValue`, `doubleValue`, + * `boolValue`); Claude Code sends several numeric fields as strings, so + * consumers still coerce rather than trusting the wire type. + * + * @param {unknown} value + * @returns {Record} + */ +export function decodeAttributes(value) { + /** @type {Record} */ + const out = {} + if (!Array.isArray(value)) return out + for (const entry of value) { + const pair = asObject(entry) + const key = stringOf(pair?.key) + if (!key) continue + out[key] = decodeAnyValue(pair?.value) + } + return out +} + +/** + * @param {unknown} value OTLP `AnyValue` + * @returns {unknown} + */ +function decodeAnyValue(value) { + const wrapper = asObject(value) + if (!wrapper) return undefined + if ('stringValue' in wrapper) return wrapper.stringValue + if ('boolValue' in wrapper) return wrapper.boolValue + if ('doubleValue' in wrapper) return numberOf(wrapper.doubleValue) + // OTLP/JSON may render an int64 as a string; keep numeric identity + // when it fits, and fall back to the raw string when it does not. + if ('intValue' in wrapper) return numberOf(wrapper.intValue) ?? wrapper.intValue + if ('arrayValue' in wrapper) { + const values = asObject(wrapper.arrayValue)?.values + return Array.isArray(values) ? values.map(decodeAnyValue) : [] + } + if ('kvlistValue' in wrapper) { + return decodeAttributes(asObject(wrapper.kvlistValue)?.values) + } + if ('bytesValue' in wrapper) return wrapper.bytesValue + return undefined +} + +/** + * @param {unknown} resource + * @returns {boolean} + */ +function resourceHasSelfMarker(resource) { + const attrs = decodeAttributes(asObject(resource)?.attributes) + const marker = attrs[SELF_MARKER_KEY] + return marker === true || marker === 'true' +} + +/** + * @param {unknown} value nanoseconds since the epoch, as a string or number + * @returns {string | undefined} + */ +function isoFromUnixNano(value) { + const nanos = typeof value === 'string' ? Number(value) + : typeof value === 'number' ? value + : undefined + if (nanos === undefined || !Number.isFinite(nanos) || nanos <= 0) return undefined + const date = new Date(Math.round(nanos / 1e6)) + return Number.isNaN(date.getTime()) ? undefined : date.toISOString() +} + +/** + * @param {unknown} value + * @returns {Record | null} + */ +function asObject(value) { + if (!value || typeof value !== 'object' || Array.isArray(value)) return null + return /** @type {Record} */ (value) +} + +/** + * @param {unknown} value + * @returns {string | undefined} + */ +function stringOf(value) { + return typeof value === 'string' && value.length > 0 ? value : undefined +} + +/** + * @param {unknown} value + * @returns {number | undefined} + */ +function numberOf(value) { + if (typeof value === 'number') return Number.isFinite(value) ? value : undefined + if (typeof value === 'string' && value.trim() !== '') { + const parsed = Number(value) + return Number.isFinite(parsed) ? parsed : undefined + } + return undefined +} diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/events_dataset.js b/hypaware-core/plugins-workspace/claude/src/telemetry/events_dataset.js new file mode 100644 index 00000000..d0fd98a4 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/events_dataset.js @@ -0,0 +1,275 @@ +// @ts-check + +import path from 'node:path' + +import { discoverCachePartitions } from '../../../../../src/core/cache/partition.js' +import { unionSources, emptySource } from 'hypaware/core/query' +import { BODY_EVENT_NAMES } from './bodies.js' +import { CONTENT_EVENT_NAMES } from './projection.js' + +/** + * @import { ColumnSpec, DatasetDataSourceContext, DatasetDiscoveryContext, DatasetRefreshResult, DatasetRegistration, QueryPartition, QueryStorageService } from '../../../../../hypaware-plugin-kernel-types.js' + * @import { ExtendedQueryStorageService } from '../../../../../src/core/cache/types.js' + * @import { ClaudeTelemetryEvent } from '../types.js' + * @import { AsyncDataSource } from 'squirreling' + */ + +const PLUGIN_NAME = '@hypaware/claude' + +/** + * The behavioral-events dataset: the first dataset `@hypaware/claude` + * owns. + * + * @ref LLP 0255#own-dataset [implements]: behavioral events get their own + * dataset, not a widening of `ai_gateway_messages` and not a route through + * `@hypaware/otel`'s generic tables + */ +export const TELEMETRY_EVENTS_DATASET = 'claude_telemetry_events' + +/** Spool partition label under the kernel cache, mirroring `@hypaware/otel`. */ +export const PARTITION_LABEL = 'all' + +/** + * The ingest signal the central forward sink POSTs this dataset's rows + * under (`/v1/ingest/claude_telemetry`). Declared so forwarding never + * falls back to the dataset name, which is not a signal the server maps. + * + * @ref LLP 0255#owned-by-claude [implements]: registration sets the source + * signal so the rows forward centrally by the same rules message rows follow + */ +export const TELEMETRY_EVENTS_SOURCE_SIGNAL = 'claude_telemetry' + +/** + * One row per event. The typed columns are the fields queries filter and + * group by; everything else the event carried rides in `attributes`. + * + * @ref LLP 0255#row-shape [implements]: hot fields typed (event name, session + * id, tool name, decision, decision source, cost), attributes JSON for the + * rest + * @type {ReadonlyArray} + */ +export const CLAUDE_TELEMETRY_EVENT_COLUMNS = Object.freeze([ + { name: 'event_name', type: 'STRING', nullable: false }, + { name: 'event_timestamp', type: 'TIMESTAMP', nullable: true }, + { name: 'session_id', type: 'STRING', nullable: true }, + { name: 'tool_name', type: 'STRING', nullable: true }, + { name: 'decision', type: 'STRING', nullable: true }, + { name: 'source', type: 'STRING', nullable: true }, + { name: 'cost_usd', type: 'DOUBLE', nullable: true }, + { name: 'attributes', type: 'JSON', nullable: true }, +]) + +/** + * On-disk spool table path under the kernel-managed cache. The listener + * writes through `ctx.storage.appendRows`; the storage service owns + * durable spool and Iceberg flush details. + * + * @param {QueryStorageService} storage + */ +export function claudeTelemetryTablePath(storage) { + return storage.cacheTablePath(TELEMETRY_EVENTS_DATASET, [PARTITION_LABEL]) +} + +/** + * Turn decoded events into `claude_telemetry_events` rows, one per + * event. + * + * The conversation half of the stream never lands here: content events + * are projected into `ai_gateway_messages` (their home), and the body + * pointer events are transport for that same projection, carrying + * nothing but a spool path. Everything else is behavior, including + * names this listener has never seen: an upstream event we do not model + * still lands with its attributes, rather than being discarded. + * + * @ref LLP 0257#failure-modes [implements]: an unrecognized event name is + * recorded with its attributes, not discarded + * @param {ClaudeTelemetryEvent[]} events + * @returns {Record[]} + */ +export function claudeTelemetryEventRows(events) { + /** @type {Record[]} */ + const rows = [] + for (const event of events) { + if (CONTENT_EVENT_NAMES.includes(event.name)) continue + if (BODY_EVENT_NAMES.includes(event.name)) continue + rows.push(rowFromEvent(event)) + } + return rows +} + +/** + * @param {ClaudeTelemetryEvent} event + * @returns {Record} + */ +function rowFromEvent(event) { + const sessionId = stringOf(event.attributes['session.id']) + const toolName = stringOf(event.attributes.tool_name) + const decision = stringOf(event.attributes.decision) + const source = stringOf(event.attributes.source) + const costUsd = numberOf(event.attributes.cost_usd) + + /** @type {Record} */ + const promoted = { + 'session.id': sessionId, + tool_name: toolName, + decision, + source, + cost_usd: costUsd, + } + + /** @type {Record} */ + const attributes = {} + for (const [key, value] of Object.entries(event.attributes)) { + if (value === undefined) continue + if (key === 'event.name' || key === 'event.timestamp') continue + // A hot key whose value did not fit its typed column (a non-string + // `decision`, say) stays in the JSON rather than vanishing: the + // split is ergonomics, not a completeness filter. + if (key in promoted && promoted[key] !== undefined) continue + attributes[key] = value + } + + return { + event_name: event.name, + event_timestamp: event.timestamp ?? null, + session_id: sessionId ?? null, + tool_name: toolName ?? null, + decision: decision ?? null, + source: source ?? null, + cost_usd: costUsd ?? null, + attributes, + } +} + +/** + * The `DatasetRegistration` `activate()` hands `ctx.query.registerDataset`. + * + * There is deliberately no pre-write dedupe on this dataset: the stream + * has one producer that POSTs each batch once, the listener writes the + * event rows only after the message-dataset write succeeded (so an + * exporter retry after a failed request re-attempts a write that never + * happened), and the one residual window - a retry after a success + * response was lost in transit - produces byte-identical rows that + * cache compaction's content-hash layer collapses. + * + * There is also no `attribution_column`: every row is a `claude` row by + * construction, so the dataset-scoped withholding rule (which withholds + * the whole dataset once its declared owner is opted out) covers it + * exactly, with no per-row column needed. + * + * @returns {DatasetRegistration} + */ +export function claudeTelemetryDatasetRegistration() { + return { + name: TELEMETRY_EVENTS_DATASET, + plugin: PLUGIN_NAME, + schema: { columns: [...CLAUDE_TELEMETRY_EVENT_COLUMNS] }, + sourceSignal: TELEMETRY_EVENTS_SOURCE_SIGNAL, + primaryTimestampColumn: 'event_timestamp', + // No `localOnlyContentColumns`: that declaration is for derived + // tables whose unprovenanced rows may AGGREGATE local-only content + // (the LLP 0105 wrapper would then null `attributes` for every + // ordinary caller, since no row here carries a `cwd` to prove + // itself with). This dataset's privacy seam is ingest instead: an + // ignored session's events are dropped before any row is written + // (LLP 0254 #policy-inline), so the rows that exist are recordable + // by construction, the same argument the message dataset's + // cwd-less rows rest on. + discoverPartitions: discoverParts, + refreshPartition: async () => /** @type {DatasetRefreshResult} */ ({ status: 'skipped', rows: 0 }), + createDataSource, + } +} + +/** + * List the spool partition (so pending rows flush during query + * settlement) plus every committed `source=` partition on disk, the + * same way the OTLP receiver's datasets discover theirs. + * + * @param {DatasetDiscoveryContext} ctx + * @returns {Promise} + */ +async function discoverParts(ctx) { + const cacheDir = ctx.cacheDir ?? '' + if (!cacheDir) return [] + + /** @type {QueryPartition[]} */ + const partitions = [] + /** @type {Set} */ + const seen = new Set() + + const spoolPath = path.join(cacheDir, 'datasets', TELEMETRY_EVENTS_DATASET, PARTITION_LABEL) + partitions.push({ + dataset: TELEMETRY_EVENTS_DATASET, + partition: { partition: PARTITION_LABEL }, + tablePath: spoolPath, + }) + seen.add(spoolPath) + + const discovered = await discoverCachePartitions(cacheDir, { datasets: [TELEMETRY_EVENTS_DATASET] }) + for (const p of discovered) { + if (seen.has(p.path)) continue + seen.add(p.path) + partitions.push({ dataset: TELEMETRY_EVENTS_DATASET, partition: p.partition, tablePath: p.path }) + } + + return partitions +} + +/** + * Union every discovered partition's source. Re-discovers from the live + * cache root so rows flushed out of the spool during settlement (after + * the initial `discoverParts`) are picked up. + * + * @param {QueryPartition[]} partitions + * @param {DatasetDataSourceContext} ctx + */ +async function createDataSource(partitions, ctx) { + const storage = /** @type {ExtendedQueryStorageService} */ (ctx.storage) + + const fresh = await discoverCachePartitions(storage.cacheRoot, { datasets: [TELEMETRY_EVENTS_DATASET] }) + + /** @type {Set} */ + const tablePaths = new Set() + for (const p of partitions) { + if (p.tablePath) tablePaths.add(p.tablePath) + } + for (const p of fresh) tablePaths.add(p.path) + + /** @type {AsyncDataSource[]} */ + const sources = [] + for (const tablePath of tablePaths) { + const source = await storage.dataSourceForTable(tablePath) + // Skip only sources KNOWN empty. icebird omits numRows when the current + // snapshot carries position deletes (a live count would need a scan), so + // treating undefined as 0 here would silently drop every partition touched + // by a retention or purge delete and blind all queries to surviving rows. + // @ref LLP 0104 [constrained-by]: position deletes leave an unknowable count that must not read as an empty partition + if (source && source.numRows !== 0) sources.push(source) + } + + if (sources.length === 0) return emptySource(CLAUDE_TELEMETRY_EVENT_COLUMNS.map((c) => c.name)) + if (sources.length === 1) return sources[0] + return unionSources(sources) +} + +/** + * @param {unknown} value + * @returns {string | undefined} + */ +function stringOf(value) { + return typeof value === 'string' && value.length > 0 ? value : undefined +} + +/** + * @param {unknown} value + * @returns {number | undefined} + */ +function numberOf(value) { + if (typeof value === 'number') return Number.isFinite(value) ? value : undefined + if (typeof value === 'string' && value.trim() !== '') { + const parsed = Number(value) + return Number.isFinite(parsed) ? parsed : undefined + } + return undefined +} diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/policy.js b/hypaware-core/plugins-workspace/claude/src/telemetry/policy.js new file mode 100644 index 00000000..1a9fb564 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/policy.js @@ -0,0 +1,110 @@ +// @ts-check + +/** + * @import { ClaudeTelemetryEvent, ClaudeTelemetrySessionVerdict, SessionContextRecord } from '../types.js' + * @import { UsagePolicyResolver } from '../../../../../src/core/usage-policy/types.js' + */ + +/** + * The verdict for a session whose cwd is not known: the SessionStart hook's + * record has not landed (or never will, for a session started without hooks). + * + * It is its own class, distinct from `full`, because "we could not ask the + * question" is not "the answer was yes". A session in this state is not + * recorded: a `.hypignore` under its cwd would have suppressed it, and writing + * first and resolving after is the fail-open window LLP 0085 exists to patch, + * which this path is not allowed to reopen. + * + * @ref LLP 0257#ingest [implements]: S10 - a session with no hook record is + * undetermined, not clean + */ +export const POLICY_UNDETERMINED = 'undetermined' + +/** + * Resolve one session's usage class from the cwd its SessionStart hook + * recorded. + * + * The resolver is the shared one every other capture seam uses, so a + * `.hypignore` dotfile and the machine-local list (LLP 0103) are both in + * scope and the more restrictive of the two wins - the listener does not get + * its own opinion about what `ignore` means. + * + * @ref LLP 0254#policy-inline [implements]: `.hypignore` and the machine-local + * list are evaluated at ingest, from the cwd the retained hook recorded + * @param {{ record: SessionContextRecord | undefined, resolver: UsagePolicyResolver }} args + * @returns {ClaudeTelemetrySessionVerdict} + */ +export function resolveSessionUsagePolicy({ record, resolver }) { + const cwd = record?.cwd + if (!cwd) return { class: POLICY_UNDETERMINED } + const policy = resolver.resolve(cwd) + /** @type {ClaudeTelemetrySessionVerdict} */ + const verdict = { class: policy.class, cwd, governedBy: policy.governedBy, declared: policy.declared } + if (policy.warn) verdict.warn = policy.warn + return verdict +} + +/** + * Split one batch of events three ways by the usage policy of the session + * each names: recorded, dropped (`ignore`), and withheld (undetermined). + * + * `local-only` is kept, exactly as the proxy projector keeps it: that class is + * enforced at the export and query seams (LLP 0070), not by refusing to record. + * + * An event that names NO session is kept, for the same reason the per-session + * opt-out keeps it: a folder policy is resolved through a session's cwd, and + * an event with no session has no cwd to resolve. Nothing conversational can + * ride out that way - the message projection skips events with no `session.id` + * outright, and the behavioral dataset does not store content events at all - + * so what is kept is a content-free counter, not somebody's prompt. + * + * @ref LLP 0254#policy-inline [implements]: the split happens before any row is + * written, so a row that must not exist is never written + * @param {ClaudeTelemetryEvent[]} events + * @param {{ verdictFor: (sessionId: string) => ClaudeTelemetrySessionVerdict }} opts + * @returns {{ + * kept: ClaudeTelemetryEvent[], + * droppedBySession: Map, + * withheldBySession: Map, + * }} + */ +export function partitionByUsagePolicy(events, { verdictFor }) { + /** @type {ClaudeTelemetryEvent[]} */ + const kept = [] + /** @type {Map} */ + const droppedBySession = new Map() + /** @type {Map} */ + const withheldBySession = new Map() + // One verdict per session per batch: the resolver caches per cwd, but the + // record lookup is a scan of the session-context tail and a batch routinely + // carries a dozen events for the same session. + /** @type {Map} */ + const verdicts = new Map() + + for (const event of events) { + const sessionId = event.attributes['session.id'] + if (typeof sessionId !== 'string' || sessionId.length === 0) { + kept.push(event) + continue + } + let verdict = verdicts.get(sessionId) + if (verdict === undefined) { + verdict = verdictFor(sessionId) + verdicts.set(sessionId, verdict) + } + const bucket = verdict.class === 'ignore' + ? droppedBySession + : verdict.class === POLICY_UNDETERMINED + ? withheldBySession + : undefined + if (bucket === undefined) { + kept.push(event) + continue + } + const existing = bucket.get(sessionId) + if (existing) existing.events.push(event) + else bucket.set(sessionId, { events: [event], verdict }) + } + + return { kept, droppedBySession, withheldBySession } +} diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/projection.js b/hypaware-core/plugins-workspace/claude/src/telemetry/projection.js new file mode 100644 index 00000000..9340b145 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/projection.js @@ -0,0 +1,392 @@ +// @ts-check + +import { BODY_EVENT_NAMES, requestBodyFacts, spooledBodyGapMessages } from './bodies.js' + +/** + * @import { AiGatewayProjectedExchange, AiGatewayProjectedMessage } from '../../../../../hypaware-plugin-kernel-types.js' + * @import { ClaudeTelemetryEvent, ClaudeTelemetrySessionFacts, SessionContextRecord, SpooledClaudeBody } from '../types.js' + */ + +/** Every row this path writes describes Claude Code talking to Anthropic. */ +const PROVIDER = 'anthropic' + +/** + * How many unclaimed `api_request` usage records to carry between + * batches. Most are claimed by the next batch's `assistant_response`; + * a turn that ends in tool calls never produces one, so the map needs a + * ceiling or a long session leaks one entry per tool round trip. + */ +export const USAGE_INDEX_LIMIT = 512 + +/** + * `conversation_source` for the OTEL path. The live proxy derives + * `claude_code` from the request User-Agent; there is no request here, + * but the producer IS Claude Code by construction (the events come from + * its own exporter), so the same value is the honest one and the two + * producers' rows stay comparable. + */ +const CONVERSATION_SOURCE = 'claude_code' + +/** + * Event names this listener turns into `ai_gateway_messages` content. + * Everything else on the stream is behavioral and belongs in + * `claude_telemetry_events` (LLP 0255), not widened into this dataset. + */ +export const CONTENT_EVENT_NAMES = Object.freeze(['user_prompt', 'assistant_response']) + +/** + * How many sessions' body-derived exchange facts (system prompt, tools) + * to carry between batches. A request body arrives in an early batch + * and the assistant response often in a later one; without carry-over + * only the rows that share a POST with the body would get the + * `system_text` and `tools` columns the proxy path stamps on every row. + * Bounded because system prompts are large and sessions are minted + * freely. + */ +export const SESSION_BODY_FACTS_LIMIT = 64 + +/** + * Split a batch of events into one projected exchange per session. + * + * The event stream is the spine: `user_prompt` and `assistant_response` + * each carry their own `message.uuid`, so a row's identity is known when + * it is written and no settlement pass has anything to repair. + * `api_request` carries no content and no uuid; it is the usage record + * for the `request_id` an `assistant_response` names, and is folded onto + * that message's `attributes.usage` rather than becoming a row of its + * own. + * + * `usageByRequestId` is owned by the caller and outlives one batch: the + * exporter flushes on a timer, so a turn's `api_request` and its + * `assistant_response` can arrive in different POSTs. + * + * Body events (`api_request_body`, `api_response_body`) join through + * `opts.spooledBodies`, keyed by the event's `body_ref`: the caller has + * already read the files (an async step this pure function cannot do). + * Their gap messages are spliced in at the body event's stream position, + * so within a session the projected order follows the body's canonical + * message ordering, which is one of the things events do not carry. + * + * @ref LLP 0252#events-first [implements]: each content event is projected + * once, from the event that carries it, with `message.uuid` as the identity + * @ref LLP 0254#identity-at-ingest [implements]: native identity, so no + * settlement enricher runs on these rows + * @param {ClaudeTelemetryEvent[]} events + * @param {{ + * clientName: string, + * usageByRequestId: Map>, + * sessionContext?: (sessionId: string) => SessionContextRecord | undefined, + * spooledBodies?: Map, + * sessionBodyFacts?: Map, + * }} opts + * @returns {AiGatewayProjectedExchange[]} + */ +export function projectClaudeTelemetryEvents(events, opts) { + /** @type {Map} */ + const bySession = new Map() + + /** @param {string} sessionId @param {ClaudeTelemetryEvent} event */ + const sessionEntry = (sessionId, event) => { + let entry = bySession.get(sessionId) + if (!entry) { + entry = { facts: sessionFacts(event), messages: [] } + bySession.set(sessionId, entry) + } + mergeSessionFacts(entry.facts, event) + return entry + } + + for (const event of events) { + const sessionId = stringAttr(event, 'session.id') + if (!sessionId) continue + + if (event.name === 'api_request') { + const requestId = stringAttr(event, 'request_id') + if (requestId) rememberUsage(opts.usageByRequestId, requestId, usageFromApiRequest(event)) + continue + } + + if (BODY_EVENT_NAMES.includes(event.name)) { + const ref = stringAttr(event, 'body_ref') + const spooled = ref ? opts.spooledBodies?.get(ref) : undefined + if (!spooled) continue + const entry = sessionEntry(sessionId, event) + mergeBodyFacts(entry.facts, spooled, sessionId, opts.sessionBodyFacts) + entry.messages.push(...spooledBodyGapMessages(spooled, { + event, + usageByRequestId: opts.usageByRequestId, + })) + continue + } + + if (!CONTENT_EVENT_NAMES.includes(event.name)) continue + + const message = messageFromEvent(event, opts.usageByRequestId) + if (!message) continue + sessionEntry(sessionId, event).messages.push(message) + } + + /** @type {AiGatewayProjectedExchange[]} */ + const projections = [] + for (const [sessionId, entry] of bySession) { + if (entry.messages.length === 0) continue + // A batch without this session's request body (the exporter splits a + // turn across POSTs) still stamps the remembered system prompt and + // tools, so the assistant rows match the proxy path's. + const remembered = opts.sessionBodyFacts?.get(sessionId) + if (remembered) { + entry.facts.systemText ??= remembered.systemText + entry.facts.tools ??= remembered.tools + } + projections.push(buildProjection({ + sessionId, + facts: entry.facts, + messages: entry.messages, + clientName: opts.clientName, + // @ref LLP 0254#hook-stays [implements]: cwd and git identity come from + // the SessionStart hook's record, not from the event attributes (the + // spike found no `workspace.host_paths` on a plain local session) + record: opts.sessionContext?.(sessionId), + })) + } + return projections +} + +/** + * @param {{ + * sessionId: string, + * facts: ClaudeTelemetrySessionFacts, + * messages: AiGatewayProjectedMessage[], + * clientName: string, + * record: SessionContextRecord | undefined, + * }} args + * @returns {AiGatewayProjectedExchange} + */ +function buildProjection({ sessionId, facts, messages, clientName, record }) { + /** @type {AiGatewayProjectedExchange} */ + const projection = { + provider: PROVIDER, + // conversation_id stays null for Claude: the session id is the + // session container, not a per-thread id. @ref LLP 0030#decision + session_id: sessionId, + conversation_source: CONVERSATION_SOURCE, + client_name: clientName, + messages, + } + if (facts.clientVersion) projection.client_version = facts.clientVersion + if (facts.entrypoint) projection.entrypoint = facts.entrypoint + if (facts.userId) projection.user_id = facts.userId + if (facts.model) projection.model = facts.model + if (facts.startedAt) projection.conversation_started_at = facts.startedAt + // @ref LLP 0252#bodies-for-gaps [implements]: the system prompt and the tool + // declarations exist only in the spooled request body; stamped + // exchange-level, exactly where the proxy path puts them. + if (facts.systemText) projection.system_text = facts.systemText + if (facts.tools !== undefined) projection.tools = /** @type {any} */ (facts.tools) + // @ref LLP 0252#consequences [implements]: `query_source` and `agent.name` + // are the attribution source on this path; parent_uuid, logical_parent_uuid, + // user_type and permission_mode are left unset and read null. + if (facts.agentName) { + projection.agent_id = facts.agentName + projection.is_sidechain = true + } + if (record?.cwd) projection.cwd = record.cwd + if (record?.git_branch) projection.git_branch = record.git_branch + // @ref LLP 0032#capture: repo identity for the graph bridge rides the same + // hook-written record the proxy and backfill producers read. + if (record?.git_remote) projection.git_remote = record.git_remote + if (record?.head_sha) projection.head_sha = record.head_sha + if (record?.repo_root) projection.repo_root = record.repo_root + + /** @type {Record} */ + const claude = {} + if (facts.querySource) claude.query_source = facts.querySource + if (facts.organizationId) claude.organization_id = facts.organizationId + if (facts.terminalType) claude.terminal_type = facts.terminalType + if (Object.keys(claude).length > 0) { + projection.attributes = /** @type {any} */ ({ claude }) + } + return projection +} + +/** + * Turn one content event into a projected message. + * + * @param {ClaudeTelemetryEvent} event + * @param {Map>} usageByRequestId + * @returns {AiGatewayProjectedMessage | undefined} + */ +function messageFromEvent(event, usageByRequestId) { + const uuid = stringAttr(event, 'message.uuid') + const promptId = stringAttr(event, 'prompt.id') + const requestId = stringAttr(event, 'request_id') + + if (event.name === 'user_prompt') { + // No `prompt` attribute means the operator did not turn on + // `OTEL_LOG_USER_PROMPTS`. There is nothing to record, so record + // nothing rather than an empty-bodied row. + const prompt = stringAttr(event, 'prompt') + if (!prompt || !uuid) return undefined + /** @type {AiGatewayProjectedMessage} */ + const message = { role: 'user', content: prompt, message_id: uuid, provider_uuid: uuid } + if (event.timestamp) message.message_created_at = event.timestamp + if (promptId) message.prompt_id = promptId + return message + } + + const response = stringAttr(event, 'response') + if (!response || !uuid) return undefined + /** @type {AiGatewayProjectedMessage} */ + const message = { role: 'assistant', content: response, message_id: uuid, provider_uuid: uuid } + if (event.timestamp) message.message_created_at = event.timestamp + if (promptId) message.prompt_id = promptId + if (requestId) message.request_id = requestId + const model = stringAttr(event, 'model') + if (model) message.model = model + // Usage lands on the assistant message, exactly where the proxy path + // puts the response's `usage` block. + const usage = requestId ? usageByRequestId.get(requestId) : undefined + if (usage) { + message.attributes = /** @type {any} */ (usage) + if (requestId) usageByRequestId.delete(requestId) + } + return message +} + +/** + * Remember one turn's usage, oldest-first evicted at the cap. `Map` + * iterates in insertion order, so the first key is the oldest. + * + * @param {Map>} index + * @param {string} requestId + * @param {Record} usage + */ +function rememberUsage(index, requestId, usage) { + index.set(requestId, usage) + while (index.size > USAGE_INDEX_LIMIT) { + const oldest = index.keys().next() + if (oldest.done) break + index.delete(oldest.value) + } +} + +/** + * Build the `attributes` block an `api_request` event contributes. + * `usage` mirrors the proxy path's shape (`cache_read_tokens` / + * `cache_write_tokens`), so a report cannot tell the producers apart; + * per-request cost and latency are net-new and sit under `claude`. + * + * @param {ClaudeTelemetryEvent} event + * @returns {Record} + */ +function usageFromApiRequest(event) { + /** @type {Record} */ + const usage = {} + const input = numberAttr(event, 'input_tokens') + const output = numberAttr(event, 'output_tokens') + const cacheRead = numberAttr(event, 'cache_read_tokens') + const cacheWrite = numberAttr(event, 'cache_creation_tokens') + if (input !== undefined) usage.input_tokens = input + if (output !== undefined) usage.output_tokens = output + if (cacheRead !== undefined) usage.cache_read_tokens = cacheRead + if (cacheWrite !== undefined) usage.cache_write_tokens = cacheWrite + + /** @type {Record} */ + const claude = {} + const costUsd = numberAttr(event, 'cost_usd') + const durationMs = numberAttr(event, 'duration_ms') + const speed = stringAttr(event, 'speed') + if (costUsd !== undefined) claude.cost_usd = costUsd + if (durationMs !== undefined) claude.duration_ms = durationMs + if (speed) claude.speed = speed + + /** @type {Record} */ + const attributes = {} + if (Object.keys(usage).length > 0) attributes.usage = usage + if (Object.keys(claude).length > 0) attributes.claude = claude + return attributes +} + +/** + * Fold a spooled request body's exchange-level fields into the session + * facts, and remember them (bounded, oldest session evicted first) so a + * later batch of the same session can still stamp them. + * + * @param {ClaudeTelemetrySessionFacts} facts + * @param {SpooledClaudeBody} spooled + * @param {string} sessionId + * @param {Map | undefined} cache + */ +function mergeBodyFacts(facts, spooled, sessionId, cache) { + const bodyFacts = requestBodyFacts(spooled) + facts.systemText ??= bodyFacts.system_text + if (facts.tools === undefined) facts.tools = bodyFacts.tools + facts.model ??= bodyFacts.model + if (cache && (facts.systemText !== undefined || facts.tools !== undefined)) { + cache.delete(sessionId) + cache.set(sessionId, { systemText: facts.systemText, tools: facts.tools }) + while (cache.size > SESSION_BODY_FACTS_LIMIT) { + const oldest = cache.keys().next() + if (oldest.done) break + cache.delete(oldest.value) + } + } +} + +/** + * @param {ClaudeTelemetryEvent} event + * @returns {ClaudeTelemetrySessionFacts} + */ +function sessionFacts(event) { + /** @type {ClaudeTelemetrySessionFacts} */ + const facts = {} + mergeSessionFacts(facts, event) + return facts +} + +/** + * Session-level identity is repeated on every event, so first-seen + * wins and later events only fill gaps. The earliest event timestamp + * seeds `conversation_started_at`. + * + * @param {ClaudeTelemetrySessionFacts} facts + * @param {ClaudeTelemetryEvent} event + */ +function mergeSessionFacts(facts, event) { + facts.clientVersion ??= stringAttr(event, 'app.version') + facts.entrypoint ??= stringAttr(event, 'app.entrypoint') + facts.userId ??= stringAttr(event, 'user.account_uuid') + facts.organizationId ??= stringAttr(event, 'organization.id') + facts.terminalType ??= stringAttr(event, 'terminal.type') + facts.querySource ??= stringAttr(event, 'query_source') + facts.agentName ??= stringAttr(event, 'agent.name') + facts.model ??= stringAttr(event, 'model') + if (event.timestamp && (facts.startedAt === undefined || event.timestamp < facts.startedAt)) { + facts.startedAt = event.timestamp + } +} + +/** + * @param {ClaudeTelemetryEvent} event + * @param {string} key + * @returns {string | undefined} + */ +function stringAttr(event, key) { + const value = event.attributes[key] + return typeof value === 'string' && value.length > 0 ? value : undefined +} + +/** + * @param {ClaudeTelemetryEvent} event + * @param {string} key + * @returns {number | undefined} + */ +function numberAttr(event, key) { + const value = event.attributes[key] + if (typeof value === 'number') return Number.isFinite(value) ? value : undefined + if (typeof value === 'string' && value.trim() !== '') { + const parsed = Number(value) + return Number.isFinite(parsed) ? parsed : undefined + } + return undefined +} diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/source.js b/hypaware-core/plugins-workspace/claude/src/telemetry/source.js new file mode 100644 index 00000000..06da2852 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/source.js @@ -0,0 +1,960 @@ +// @ts-check + +import { Attr, getActiveSpan, withSpan } from '../../../../../src/core/observability/index.js' +import { readObservabilityEnv } from '../../../../../src/core/observability/env.js' +import { SESSION_IGNORE_ROUTE, createControlHandler } from '../../../../../src/core/control/session_ignore.js' +import { resolveLiveSourceListenPortFromStatus } from '../../../../../src/core/daemon/status.js' +import { createOtlpJsonServer, listenAndResolve } from '../../../../../src/core/otlp/server.js' +import { createUsagePolicyResolver } from '../../../../../src/core/usage-policy/index.js' +import { createSessionContextReader, pickLatestMatching } from '../session_context.js' +import { deleteSpooledBodies, deleteSpooledBodiesForEvents, loadSpooledBodies } from './bodies.js' +import { partitionByUsagePolicy, resolveSessionUsagePolicy } from './policy.js' +import { flattenClaudeTelemetryEvents, flattenClaudeTelemetryMetrics } from './events.js' +import { + CLAUDE_TELEMETRY_EVENT_COLUMNS, + claudeTelemetryEventRows, + claudeTelemetryTablePath, +} from './events_dataset.js' +import { projectClaudeTelemetryEvents } from './projection.js' +import { + DEFAULT_SPOOL_MAX_BYTES, + claudeBodySpoolDir, + enforceClaudeBodySpoolCap, + tightenClaudeBodySpool, +} from './spool.js' + +/** + * @import { Server } from 'node:http' + * @import { AiGatewayCapability, PluginActivationContext, SourceStatus, StartedSource } from '../../../../../hypaware-plugin-kernel-types.js' + * @import { OtlpRequest } from '../../../../../src/core/otlp/types.js' + * @import { UsagePolicyResolver } from '../../../../../src/core/usage-policy/types.js' + * @import { BatchSuppressionTally, ClaudeTelemetryEvent, ClaudeTelemetryListenerState, SessionContextRecord } from '../types.js' + */ + +const PLUGIN_NAME = '@hypaware/claude' + +/** Kernel source name. Registered by `@hypaware/claude`, not by `@hypaware/otel`. */ +export const CLAUDE_TELEMETRY_SOURCE = 'claude-telemetry' + +/** What this listener calls itself on the wire banner and in bind errors. */ +const LISTENER_NAME = 'hypaware/claude-telemetry' + +/** + * Loopback only. The endpoint attach writes into the settings `env` + * block is `http://127.0.0.1:` (LLP 0258 #env-keys), and a + * listener that carried raw prompt text off the loopback interface + * would be a capture surface nobody asked for. + */ +const DEFAULT_HOST = '127.0.0.1' + +/** + * Own port, next to `@hypaware/otel`'s 4318 and separate from the + * gateway's. One listener per payload dialect: the OTLP receiver keeps + * flattening generic logs/traces/metrics, and this one reads Claude + * Code's event vocabulary. + * @ref LLP 0257#registration [implements]: its own port, separate from the otel + * receiver and from the gateway + */ +export const DEFAULT_TELEMETRY_PORT = 4319 + +/** + * Claude Code's exporter is configured with both `OTEL_LOGS_EXPORTER` + * and `OTEL_METRICS_EXPORTER` (LLP 0258 #env-keys), so it POSTs to + * `/v1/metrics` too. Both halves of the stream are consumed: log events + * feed the message projection plus the behavioral dataset, and metric + * data points land in `claude_telemetry_events` under LLP 0255. + */ +const SERVED_SIGNALS = /** @type {const} */ (['logs', 'metrics']) + +/** + * How often the daemon re-enforces the spool's byte cap. The listener + * deletes what it projects, so under normal flow the sweep finds + * nothing; the interval exists for the window where Claude Code is + * writing bodies and nothing is consuming them (a misdirected exporter, + * a wedged storage service), which is exactly when nobody else would + * notice the directory growing. + */ +const SPOOL_SWEEP_INTERVAL_MS = 60_000 + +/** + * Build the `SourceContribution.start` callback for the Claude + * telemetry listener. + * + * The source owns one HTTP listener and writes into + * `ai_gateway_messages` through the gateway capability, so the dataset + * keeps one owner and the OTEL rows cannot drift from the proxy's. + * + * @ref LLP 0257#registration [implements]: a listener source contributed by + * `@hypaware/claude` through the kernel source registry + * @param {{ + * gateway: AiGatewayCapability, + * clientName: string, + * stateFile: string, + * localOnlyListPath?: string, + * }} deps + */ +export function createStartClaudeTelemetrySource(deps) { + /** + * @param {PluginActivationContext} ctx + * @returns {Promise} + */ + return async function startClaudeTelemetrySource(ctx) { + const listen = readListenConfig(ctx) + const spool = readSpoolConfig(ctx) + /** @type {ClaudeTelemetryListenerState} */ + const state = { + rowsWritten: 0, + rowsSkipped: 0, + telemetryRowsWritten: 0, + eventsReceived: 0, + eventsDropped: 0, + eventsUndetermined: 0, + lastEventAt: undefined, + lastError: undefined, + listenFallbackFrom: undefined, + spoolBytes: 0, + bodiesProjected: 0, + bodiesDeleted: 0, + bodiesDropped: 0, + bodiesEvicted: 0, + bodiesMissing: 0, + bodiesUnparseable: 0, + } + + // The same per-session opt-out the gateway keeps: an in-memory set that + // dies with the process, written through the identical control route and + // matched verbatim against the `session.id` the events carry (LLP 0066 + // R5). Nothing about it touches disk. + // @ref LLP 0256#in-memory-only [implements]: no new on-disk contract; the + // durable expressions of the same intent stay `.hypignore` and the + // machine-local list + /** @type {Set} */ + const ignoredSessions = new Set() + + // The spool exists whether or not this daemon was up when attach ran: + // Claude Code starts writing bodies the moment a session launches with + // the attach-written env, so the listener repairs permissions and + // enforces the cap on every start, then keeps enforcing on a timer for + // the window where bodies arrive but nothing consumes them. + // + // Repair, never create: attach is the write that mints this directory, + // because it is the same write that tells Claude Code to put bodies in + // it. A daemon that created it regardless would leave a raw-prompt + // directory on every install that never attached this client, in whatever + // HYP_HOME the activation context resolved. + // @ref LLP 0253#spool-location [implements]: owner-only under the HypAware + // home, tightened here even when Claude Code created it first + try { + await tightenClaudeBodySpool(spool.dir) + } catch (err) { + ctx.log.warn('claude.telemetry.spool_unavailable', { + [Attr.PLUGIN]: PLUGIN_NAME, + spool_dir: spool.dir, + error: err instanceof Error ? err.message : String(err), + }) + } + const sweepSpool = async () => { + try { + const swept = await enforceClaudeBodySpoolCap(spool.dir, spool.maxBytes) + state.spoolBytes = swept.spoolBytes + if (swept.evictedCount > 0) { + state.bodiesEvicted += swept.evictedCount + // A machine that is routinely evicting is losing detail to the + // backfill path; the count is what makes that visible. + // @ref LLP 0253#byte-cap [implements]: eviction is logged with a count + ctx.log.warn('claude.telemetry.spool_evicted', { + [Attr.PLUGIN]: PLUGIN_NAME, + [Attr.COMPONENT]: 'sources', + [Attr.OPERATION]: 'spool_sweep', + spool_dir: spool.dir, + evicted_count: swept.evictedCount, + evicted_bytes: swept.evictedBytes, + spool_bytes: swept.spoolBytes, + spool_max_bytes: spool.maxBytes, + }) + } + } catch (err) { + ctx.log.warn('claude.telemetry.spool_sweep_failed', { + [Attr.PLUGIN]: PLUGIN_NAME, + spool_dir: spool.dir, + error: err instanceof Error ? err.message : String(err), + }) + } + } + // The one-shot sweep runs whether or not the bind below succeeds: bodies + // already on disk are over the cap regardless. The repeating one does not + // get armed until there is a listener behind it, because `stop()` is the + // only thing that clears it and a start that throws never returns a handle + // to call `stop()` on - an armed timer would then keep scanning the spool + // every minute, for the life of the daemon, on behalf of a source that + // does not exist. + await sweepSpool() + + const readSessionContext = createSessionContextReader(deps.stateFile, (err) => { + ctx.log.warn('claude.telemetry.session_context_unreadable', { + [Attr.PLUGIN]: PLUGIN_NAME, + error: err instanceof Error ? err.message : String(err), + }) + }) + + // One resolver per listener (per daemon run), like the projector's: the + // per-cwd cache rides the source's lifetime so the ingest path adds no + // unbounded fs work. `localOnlyListPath` is threaded from the plugin's + // SHARED state root, which is where the machine-local list actually lives; + // without it the resolver would see `.hypignore` dotfiles only and a + // `--private` directory would record here after being dropped everywhere + // else. + // @ref LLP 0254#policy-inline [implements]: the same shared resolver every + // other capture seam uses, so `.hypignore` and the machine-local list + // both reach the OTEL path + const resolver = createUsagePolicyResolver({ localOnlyListPath: deps.localOnlyListPath }) + + /** + * Held across batches: the exporter flushes on a timer, so a turn's + * `api_request` (which carries the tokens) and its + * `assistant_response` (which carries the uuid the row is keyed by) + * can arrive in different POSTs. Bounded so a stream of requests + * that never produce an assistant response cannot grow it forever. + */ + /** @type {Map>} */ + const usageByRequestId = new Map() + + /** + * Also held across batches: a session's system prompt and tool + * declarations arrive once, in the request body, while the rows they + * belong on keep arriving for the session's lifetime. + */ + /** @type {Map} */ + const sessionBodyFacts = new Map() + + const handler = makeReceiveHandler({ + ctx, + deps, + state, + usageByRequestId, + sessionBodyFacts, + readSessionContext, + resolver, + spoolDir: spool.dir, + ignoredSessions, + }) + const server = createOtlpJsonServer({ + name: LISTENER_NAME, + handler: { handle: handler }, + signals: [...SERVED_SIGNALS], + // The same `/_hypaware/ignore/session` route the gateway proxy hosts, + // over this listener's own set: with Claude Code traffic no longer on + // the gateway's wire, "don't record this conversation" has to reach + // the recorder that now writes the rows. + // @ref LLP 0256#control-route-on-listener [implements]: same shape, + // verbs, and reply as the gateway's, via the shared handler + onControlRequest: createControlHandler({ + ignoredSessions, + log: ctx.log, + logEvent: 'claude.telemetry.control.ignore_session', + logFields: { [Attr.PLUGIN]: PLUGIN_NAME, [Attr.COMPONENT]: 'sources' }, + }), + }) + const bound = await bindWithFallback({ server, listen, log: ctx.log, state }) + const sweepTimer = setInterval(sweepSpool, SPOOL_SWEEP_INTERVAL_MS) + sweepTimer.unref?.() + + // When this listener came up, published so `hyp status` can tell "nothing + // has arrived yet because the daemon restarted a minute ago" from "nothing + // has arrived for a day". `lastEventAt` lives only in this object, so every + // restart republishes `last_event_at: null` however long capture has been + // healthy, and the capture-health baseline would otherwise fall back to an + // attach timestamp that can be weeks old. + // @ref LLP 0257#status-and-health [implements]: the gap is measured from a moment capture was actually supposed to be running + const startedAt = new Date().toISOString() + + const span = getActiveSpan() + span?.setAttribute('listen_host', bound.host) + span?.setAttribute('listen_port', bound.port) + ctx.log.info('claude.telemetry.listener_started', { + [Attr.PLUGIN]: PLUGIN_NAME, + listen_host: bound.host, + listen_port: bound.port, + }) + + return { + async status() { + /** @type {SourceStatus} */ + const status = { + state: 'ready', + rowsWritten: state.rowsWritten, + details: { + listen_host: bound.host, + listen_port: bound.port, + // Says "I host the session-ignore route here", which is how + // `hyp session ignore` discovers this recorder beside the + // gateway without the client-agnostic verb naming any plugin. + // @ref LLP 0256#cli-posts-to-both [implements]: offering the route + // is advertised by the recorder itself + control_routes: [SESSION_IGNORE_ROUTE], + events_received: state.eventsReceived, + rows_skipped: state.rowsSkipped, + // Behavioral rows, counted apart from `rowsWritten` (message + // rows) so a capture gap in either dataset is visible alone. + telemetry_rows_written: state.telemetryRowsWritten, + // @ref LLP 0257#status-and-health [implements]: the status details + // carry the spool's byte size and eviction count. + spool_bytes: state.spoolBytes, + bodies_projected: state.bodiesProjected, + bodies_evicted: state.bodiesEvicted, + // The live opt-out surface, mirroring the gateway source's + // details: the set size plus what enforcing it dropped. + // @ref LLP 0066#ephemeral: an active session drop is visible in + // status, not only in logs + ignored_sessions: ignoredSessions.size, + events_dropped: state.eventsDropped, + bodies_dropped: state.bodiesDropped, + // A capture gap, not a policy outcome: these events named a session + // whose cwd nothing had recorded yet, so there was no verdict to + // record them under. Published because a machine whose hook is not + // installed would otherwise look idle rather than blind. + // @ref LLP 0257#ingest [implements]: S10 - undetermined is its own + // visible state, not silence + events_undetermined: state.eventsUndetermined, + // Null before the first event rather than absent: the key's + // presence is how the capture-health reader recognizes this + // snapshot as the telemetry listener's (the `control_routes` + // self-advertisement pattern), and "attached but nothing ever + // arrived" is exactly the state that comparison must be able + // to see. + // @ref LLP 0257#status-and-health [implements]: the last event seen, published for the hyp status comparison + last_event_at: state.lastEventAt ?? null, + // Beside it, and for the same reader: the window this listener has + // actually been able to capture in. + listener_started_at: startedAt, + ...(state.listenFallbackFrom !== undefined + ? { listen_fallback_from: state.listenFallbackFrom } + : {}), + }, + } + if (state.lastError) status.lastError = state.lastError + return status + }, + + async stop() { + clearInterval(sweepTimer) + await new Promise((resolve, reject) => { + server.close((err) => (err ? reject(err) : resolve(undefined))) + server.closeIdleConnections?.() + server.closeAllConnections?.() + }) + }, + } + } +} + +/** + * Wrap one decoded OTLP request in a `claude.telemetry.receive` span, + * read the spooled bodies its events reference, project everything, and + * write the rows. Consumed body files are deleted only after the writes + * succeeded: a write failure becomes an HTTP error the exporter + * retries, and the retried batch re-reads the same files. + * + * A throw here becomes an HTTP 500 the exporter will retry, so + * everything recoverable is handled: an unparseable envelope yields zero + * events, an event this listener does not model is skipped, a missing + * or refused body ref is counted, and only a genuine write failure + * propagates. + * + * @param {{ + * ctx: PluginActivationContext, + * deps: { gateway: AiGatewayCapability, clientName: string, stateFile: string }, + * state: ClaudeTelemetryListenerState, + * usageByRequestId: Map>, + * sessionBodyFacts: Map, + * readSessionContext: () => Promise, + * resolver: UsagePolicyResolver, + * spoolDir: string, + * ignoredSessions: Set, + * }} args + * @returns {(req: OtlpRequest) => Promise} + */ +function makeReceiveHandler({ ctx, deps, state, usageByRequestId, sessionBodyFacts, readSessionContext, resolver, spoolDir, ignoredSessions }) { + /** + * Suppress one session's events at ingest: delete its spooled bodies + * WITHOUT reading them, count it, and emit one drop signal so the audit + * trail matches the proxy path's. + * + * The deletion is the half that makes an opt-out mean what it says. A skip + * would leave the content of exactly the session the user asked us not to + * keep sitting in our own directory until the cap evicted it. + * + * @ref LLP 0253#delete-on-drop [implements]: a dropped session's bodies are + * deleted, never merely skipped + * @ref LLP 0256#bodies-deleted [implements]: the same duty for the + * per-session opt-out + * @param {{ + * sessionId: string, + * events: ClaudeTelemetryEvent[], + * policySource: string, + * withheld?: boolean, + * fields?: Record, + * tally: BatchSuppressionTally, + * }} args + */ + async function suppressSession({ sessionId, events, policySource, withheld = false, fields = {}, tally }) { + const removal = await deleteSpooledBodiesForEvents(events, { spoolDir }) + tally.bodiesDropped += removal.deleted + state.bodiesDropped += removal.deleted + if (withheld) { + tally.eventsUndetermined += events.length + state.eventsUndetermined += events.length + } else { + tally.eventsDropped += events.length + state.eventsDropped += events.length + } + for (const ref of removal.refused) { + ctx.log.warn('claude.telemetry.body_ref_refused', { + [Attr.PLUGIN]: PLUGIN_NAME, + error_kind: 'body_ref_outside_spool', + body_ref: ref, + }) + } + // Warn for a withheld session, info for a policy that answered: the first + // is a capture gap an operator can close (install the hook), the second is + // the system doing what it was told. + ctx.log[withheld ? 'warn' : 'info']('claude.telemetry.usage_policy_drop', { + [Attr.PLUGIN]: PLUGIN_NAME, + [Attr.COMPONENT]: 'sources', + [Attr.OPERATION]: 'usage_policy_drop', + policy_source: policySource, + session_id: sessionId, + events_dropped: events.length, + bodies_deleted: removal.deleted, + ...fields, + }) + } + + /** + * Enforce the per-session opt-out (LLP 0066 / LLP 0256) on one batch. + * + * @param {Map} droppedBySession + * @param {BatchSuppressionTally} tally + */ + async function dropIgnoredSessions(droppedBySession, tally) { + for (const [sessionId, sessionEvents] of droppedBySession) { + await suppressSession({ + sessionId, + events: sessionEvents, + policySource: 'session_opt_out', + tally, + }) + } + } + + /** + * Enforce the folder usage policy on one batch, INLINE, before anything is + * read from the spool and before any row is written. + * + * Three outcomes per session: an `ignore` cwd is dropped with its bodies, a + * session whose cwd nothing has recorded is withheld the same way (no + * verdict exists, so there is nothing to record it under), and everything + * else is returned to be projected. `local-only` is deliberately in that + * last group: it is enforced at the export and query seams, not by refusing + * to record. + * + * There is no second look at flush. That is the point: the proxy path writes + * provisionally and lets settlement drop a late-resolved `ignore` row + * (LLP 0085), and this path has no such window to patch because the verdict + * is in hand before the write. + * + * @ref LLP 0254#policy-inline [implements]: the check runs at ingest with cwd + * in hand, so the fail-open window cannot reappear + * @ref LLP 0254#scope [constrained-by]: LLP 0027 / LLP 0085 stay in force for + * the proxy and backfill producers; only this path settles at ingest + * @param {ClaudeTelemetryEvent[]} events + * @param {{ records: SessionContextRecord[], tally: BatchSuppressionTally }} args + * @returns {Promise} the events cleared to be written + */ + async function applyUsagePolicy(events, { records, tally }) { + const split = partitionByUsagePolicy(events, { + verdictFor: (sessionId) => resolveSessionUsagePolicy({ + record: pickLatestMatching(records, { sessionId }), + resolver, + }), + }) + for (const [sessionId, entry] of split.droppedBySession) { + await suppressSession({ + sessionId, + events: entry.events, + policySource: 'usage_policy', + tally, + fields: { + // The governing file, as the proxy projector reports it, so one query + // answers "what suppressed this" across both producers. The cwd + // itself is not logged on either path. + governed_by: entry.verdict.governedBy ?? null, + declared: entry.verdict.declared ?? null, + ...(entry.verdict.warn ? { warn: entry.verdict.warn } : {}), + }, + }) + } + for (const [sessionId, entry] of split.withheldBySession) { + await suppressSession({ + sessionId, + events: entry.events, + policySource: 'undetermined_cwd', + withheld: true, + tally, + fields: { + // Names the recovery path in the signal itself: the content is still + // in the Claude Code transcript, where `hyp backfill claude` reads it + // with the cwd resolved per session. + recovery: 'transcript_backfill', + }, + }) + } + return split.kept + } + + /** + * Publish one batch's suppression counts on its receive span. Set only when + * non-zero, so a routine batch's span stays free of noise fields. + * + * @param {{ setAttribute(key: string, value: unknown): unknown }} span + * @param {BatchSuppressionTally} tally + */ + function recordSuppression(span, tally) { + if (tally.eventsDropped > 0) span.setAttribute('events_dropped', tally.eventsDropped) + if (tally.bodiesDropped > 0) span.setAttribute('bodies_dropped', tally.bodiesDropped) + if (tally.eventsUndetermined > 0) { + span.setAttribute('events_undetermined', tally.eventsUndetermined) + } + } + + return async function handle(req) { + await withSpan( + 'claude.telemetry.receive', + { + [Attr.COMPONENT]: 'sources', + [Attr.PLUGIN]: PLUGIN_NAME, + [Attr.OPERATION]: 'claude.telemetry.receive', + hyp_source: CLAUDE_TELEMETRY_SOURCE, + signal: req.signal, + payload_bytes: req.payloadBytes, + status: 'ok', + }, + async (span) => { + /** @type {BatchSuppressionTally} */ + const tally = { eventsDropped: 0, eventsUndetermined: 0, bodiesDropped: 0 } + + // Metrics ride the same exporter config; the message dataset has + // nothing to learn from them, but the behavioral dataset does + // (cost and activity counters), so they take the short path: + // flatten, record, done - no bodies, no projection. + if (req.signal === 'metrics') { + const allMetricEvents = flattenClaudeTelemetryMetrics(req.data) + span.setAttribute('event_count', allMetricEvents.length) + span.setAttribute('row_count', 0) + if (allMetricEvents.length === 0) { + span.setAttribute('telemetry_row_count', 0) + return + } + state.eventsReceived += allMetricEvents.length + for (const event of allMetricEvents) { + if (event.timestamp && (state.lastEventAt === undefined || event.timestamp > state.lastEventAt)) { + state.lastEventAt = event.timestamp + } + } + // The opt-out covers the behavioral record too: a metric data + // point names its session, so it is droppable on the same key. + // @ref LLP 0256#control-route-on-listener [implements]: ingest drops + // by session id on every signal this listener serves + const metricSplit = partitionIgnoredSessionEvents(allMetricEvents, ignoredSessions) + if (metricSplit.droppedBySession.size > 0) { + await dropIgnoredSessions(metricSplit.droppedBySession, tally) + } + // And so does the folder policy: a cost counter names its session + // and its model, which is attribution for a directory the user asked + // us to leave alone. + const metricEvents = await applyUsagePolicy(metricSplit.kept, { + records: await readSessionContext(), + tally, + }) + recordSuppression(span, tally) + if (metricEvents.length === 0) { + span.setAttribute('telemetry_row_count', 0) + return + } + const written = await recordTelemetryEvents(metricEvents, { ctx, state, span }) + span.setAttribute('telemetry_row_count', written) + ctx.log.info('claude.telemetry.batch', { + [Attr.PLUGIN]: PLUGIN_NAME, + signal: req.signal, + event_count: metricEvents.length, + telemetry_rows_written: written, + }) + return + } + + const allEvents = flattenClaudeTelemetryEvents(req.data) + span.setAttribute('event_count', allEvents.length) + if (allEvents.length === 0) { + span.setAttribute('row_count', 0) + return + } + state.eventsReceived += allEvents.length + for (const event of allEvents) { + if (event.timestamp && (state.lastEventAt === undefined || event.timestamp > state.lastEventAt)) { + state.lastEventAt = event.timestamp + } + } + + // The per-session opt-out, enforced at ingest BEFORE the spool is + // read: a dropped session's events project nothing into either + // dataset, and its body files are deleted rather than skipped, so + // the transport works AND the content goes. + // @ref LLP 0256#bodies-deleted [implements] + const split = partitionIgnoredSessionEvents(allEvents, ignoredSessions) + if (split.droppedBySession.size > 0) { + await dropIgnoredSessions(split.droppedBySession, tally) + } + + // Then the folder policy, on the same batch, still before the spool is + // read and before anything is written. Both gates run ahead of every + // write on this path, which is what leaves no window for a verdict to + // arrive after the data. + // @ref LLP 0254#policy-inline [implements]: `.hypignore` and the + // machine-local list decide at ingest, from the hook's cwd + const records = await readSessionContext() + const events = await applyUsagePolicy(split.kept, { records, tally }) + recordSuppression(span, tally) + if (events.length === 0) { + span.setAttribute('row_count', 0) + span.setAttribute('telemetry_row_count', 0) + return + } + + // @ref LLP 0257#ingest [implements]: body files named by `body_ref` + // are read for the gap fields, then deleted after the write below. + const spooled = await loadSpooledBodies(events, { spoolDir }) + state.bodiesMissing += spooled.missing + state.bodiesUnparseable += spooled.unparseable + span.setAttribute('body_count', spooled.bodies.size) + if (spooled.unparseable > 0) { + span.setAttribute('bodies_unparseable', spooled.unparseable) + ctx.log.warn('claude.telemetry.body_unparseable', { + [Attr.PLUGIN]: PLUGIN_NAME, + error_kind: 'body_unparseable', + body_count: spooled.unparseable, + }) + } + for (const ref of spooled.refused) { + ctx.log.warn('claude.telemetry.body_ref_refused', { + [Attr.PLUGIN]: PLUGIN_NAME, + error_kind: 'body_ref_outside_spool', + body_ref: ref, + }) + } + + // The same records the policy gate decided on, so the cwd a row is + // stamped with is the cwd its verdict was resolved from. + const projections = projectClaudeTelemetryEvents(events, { + clientName: deps.clientName, + usageByRequestId, + sessionContext: (sessionId) => pickLatestMatching(records, { sessionId }), + spooledBodies: spooled.bodies, + sessionBodyFacts, + }) + span.setAttribute('session_count', projections.length) + + let rowsWritten = 0 + let rowsSkipped = 0 + try { + for (const projection of projections) { + // @ref LLP 0252#projection-unchanged [implements]: the same + // projected-exchange path the proxy and backfill producers use, so + // `part_id` dedupe absorbs the overlap between them. + const result = await deps.gateway.recordProjectedExchange(projection, { + gatewayAttributes: { gateway: { source: 'otel' } }, + }) + rowsWritten += result.rowsWritten + rowsSkipped += result.rowsSkipped + } + } catch (err) { + state.lastError = err instanceof Error ? err.message : String(err) + span.setAttribute('error_kind', 'dataset_write') + span.setAttribute('row_count', rowsWritten) + ctx.log.error('claude.telemetry.write_failed', { + [Attr.PLUGIN]: PLUGIN_NAME, + event_count: events.length, + error: state.lastError, + }) + throw err + } + + state.rowsWritten += rowsWritten + state.rowsSkipped += rowsSkipped + + // The behavioral half of the batch, written AFTER the message + // rows: a failure here becomes an HTTP error the exporter + // retries, and on that retry the message rows dedupe by + // `part_id` while this write (which never happened) is + // re-attempted - the reverse order would duplicate behavioral + // rows on every message-write retry. + // @ref LLP 0255#own-dataset [implements]: behavioral events land in + // `claude_telemetry_events`, next to (not inside) the message rows + const telemetryRowsWritten = await recordTelemetryEvents(events, { ctx, state, span }) + span.setAttribute('telemetry_row_count', telemetryRowsWritten) + + // Projected, then deleted: the writes above succeeded, so nothing + // will ever need these files again. + // @ref LLP 0252#project-then-delete [implements]: deletion is the + // normal end of a body's life, not a cleanup pass + if (spooled.consumedFiles.length > 0) { + const deleted = await deleteSpooledBodies(spooled.consumedFiles) + state.bodiesProjected += spooled.bodies.size + state.bodiesDeleted += deleted + state.spoolBytes = Math.max(0, state.spoolBytes - spooled.consumedBytes) + span.setAttribute('bodies_projected', spooled.bodies.size) + span.setAttribute('bodies_deleted', deleted) + } + + span.setAttribute('row_count', rowsWritten) + span.setAttribute('rows_skipped', rowsSkipped) + ctx.log.info('claude.telemetry.batch', { + [Attr.PLUGIN]: PLUGIN_NAME, + event_count: events.length, + session_count: projections.length, + rows_written: rowsWritten, + rows_skipped: rowsSkipped, + telemetry_rows_written: telemetryRowsWritten, + bodies_projected: spooled.bodies.size, + bodies_missing: spooled.missing, + }) + }, + { component: 'plugin.claude' } + ) + } +} + +/** + * Split one batch by the in-memory ignored-session set: events whose + * `session.id` is in the set are dropped, everything else is kept. + * + * The match key is the raw `session.id` the event carries, compared + * verbatim against the raw token the control route stored - the same R5 + * discipline the gateway's drop applies, so `hyp session ignore` reaches + * both recorders with one resolved id. An event that names NO session is + * kept: the set holds exact keys, and dropping what cannot be matched + * would suppress rows nobody opted out. + * + * @ref LLP 0066#requirements: R5 - the match key is the session_id the + * recorder resolves and stamps, verbatim + * @param {ClaudeTelemetryEvent[]} events + * @param {Set} ignoredSessions + * @returns {{ kept: ClaudeTelemetryEvent[], droppedBySession: Map }} + */ +export function partitionIgnoredSessionEvents(events, ignoredSessions) { + /** @type {ClaudeTelemetryEvent[]} */ + const kept = [] + /** @type {Map} */ + const droppedBySession = new Map() + if (ignoredSessions.size === 0) return { kept: events.slice(), droppedBySession } + for (const event of events) { + const sessionId = event.attributes['session.id'] + if (typeof sessionId === 'string' && ignoredSessions.has(sessionId)) { + const bucket = droppedBySession.get(sessionId) + if (bucket) bucket.push(event) + else droppedBySession.set(sessionId, [event]) + } else { + kept.push(event) + } + } + return { kept, droppedBySession } +} + +/** + * Write one batch's behavioral rows into `claude_telemetry_events`. + * Content and body-pointer events yield no rows (`claudeTelemetryEventRows` + * filters them), so a purely conversational batch writes nothing here. + * + * A failure is handled exactly like a message-dataset write failure: + * counted on the state, marked on the span, logged, and re-thrown so + * the transport answers with an error the exporter retries. + * + * @ref LLP 0257#outputs [implements]: one row per event, hot fields typed, + * the remainder in the attributes JSON column + * @param {ClaudeTelemetryEvent[]} events + * @param {{ + * ctx: PluginActivationContext, + * state: ClaudeTelemetryListenerState, + * span: { setAttribute(key: string, value: unknown): unknown }, + * }} args + * @returns {Promise} rows written + */ +async function recordTelemetryEvents(events, { ctx, state, span }) { + const rows = claudeTelemetryEventRows(events) + if (rows.length === 0) return 0 + try { + await ctx.storage.appendRows( + claudeTelemetryTablePath(ctx.storage), + [...CLAUDE_TELEMETRY_EVENT_COLUMNS], + rows + ) + } catch (err) { + state.lastError = err instanceof Error ? err.message : String(err) + span.setAttribute('error_kind', 'dataset_write') + ctx.log.error('claude.telemetry.write_failed', { + [Attr.PLUGIN]: PLUGIN_NAME, + dataset: 'claude_telemetry_events', + event_count: events.length, + error: state.lastError, + }) + throw err + } + state.telemetryRowsWritten += rows.length + return rows.length +} + +/** + * Bind the listener, falling back to an ephemeral port when the DEFAULT + * port is taken. An address the operator wrote down fails loudly + * instead; a default that happens to collide must not stop the daemon, + * because attach reads the bound port off the source status anyway. + * + * @ref LLP 0114#explicit-listen-fails-loudly [constrained-by]: only the + * unconfigured default falls back + * @param {{ + * server: Server, + * listen: { host: string, port: number, portConfigured: boolean }, + * log: PluginActivationContext['log'], + * state: { listenFallbackFrom: number | undefined }, + * }} args + */ +async function bindWithFallback({ server, listen, log, state }) { + try { + return await listenAndResolve(server, listen.host, listen.port, LISTENER_NAME) + } catch (err) { + const code = err && /** @type {NodeJS.ErrnoException} */ (err).code + if (listen.portConfigured || code !== 'EADDRINUSE') throw err + log.warn('claude.telemetry.default_port_taken', { + [Attr.PLUGIN]: PLUGIN_NAME, + listen_port: listen.port, + }) + state.listenFallbackFrom = listen.port + return listenAndResolve(server, listen.host, 0, LISTENER_NAME) + } +} + +/** + * The listener port `otel`-mode attach writes into + * `env.OTEL_EXPORTER_OTLP_ENDPOINT`. + * + * Three rungs, in trust order. The running daemon's bound port wins: it is the + * only place the truth lives once the default-port fallback in + * {@link bindWithFallback} has moved the listener, and the source status is + * where that promise was made. With no live daemon, a configured fixed port is + * the address the operator stated. Otherwise the well-known default: the same + * port the next daemon start will try first, so an attach that ran before the + * first daemon start still points where the listener will appear. A configured + * `0` (dynamic) has no knowable port until a daemon publishes one, so it reads + * as unconfigured here. + * + * @ref LLP 0258#env-keys [constrained-by]: the endpoint's port must be the + * listener's real one, or the whole env block captures nothing + * @param {{ stateRoot: string, config: unknown }} args + * @returns {number} + */ +export function resolveAttachTelemetryPort({ stateRoot, config }) { + const live = resolveLiveSourceListenPortFromStatus({ + stateRoot, + sourceName: CLAUDE_TELEMETRY_SOURCE, + }) + if (live !== undefined) return live + + const raw = /** @type {Record} */ ( + config && typeof config === 'object' && !Array.isArray(config) ? config : {} + ) + const telemetry = raw.telemetry + const slice = telemetry && typeof telemetry === 'object' && !Array.isArray(telemetry) + ? /** @type {Record} */ (telemetry) + : {} + const portRaw = slice.listen_port + if (typeof portRaw === 'number' && Number.isInteger(portRaw) && portRaw >= 1 && portRaw <= 65535) { + return portRaw + } + return DEFAULT_TELEMETRY_PORT +} + +/** + * Resolve where the body spool lives and how large it may grow. The + * directory is fixed under the HypAware home (attach, detach, and + * `hyp purge` all derive the same path); only the byte cap is config, + * `telemetry.spool_max_bytes`, defaulting to 512 MB. A mistyped cap + * falls back to the default and warns, matching `readListenConfig`. + * + * @ref LLP 0253#byte-cap [implements]: the cap is one config value an operator + * can lower on a small disk + * @param {PluginActivationContext} ctx + * @returns {{ dir: string, maxBytes: number }} + */ +export function readSpoolConfig(ctx) { + const dir = claudeBodySpoolDir(readObservabilityEnv(ctx.env).hypHome) + const config = /** @type {Record} */ (ctx.config ?? {}) + const telemetry = config.telemetry + const slice = telemetry && typeof telemetry === 'object' && !Array.isArray(telemetry) + ? /** @type {Record} */ (telemetry) + : {} + + let maxBytes = DEFAULT_SPOOL_MAX_BYTES + const raw = slice.spool_max_bytes + if (typeof raw === 'number' && Number.isInteger(raw) && raw >= 1) { + maxBytes = raw + } else if (raw !== undefined) { + ctx.log.warn('claude.telemetry.config_invalid', { + [Attr.PLUGIN]: PLUGIN_NAME, + key: 'telemetry.spool_max_bytes', + value_type: typeof raw, + }) + } + return { dir, maxBytes } +} + +/** + * Read `telemetry.listen_host` / `telemetry.listen_port` out of the + * plugin's config slice. Mistyped values fall back to the defaults and + * warn, matching the OTLP receiver's behavior. + * + * @param {PluginActivationContext} ctx + * @returns {{ host: string, port: number, portConfigured: boolean }} + */ +export function readListenConfig(ctx) { + const config = /** @type {Record} */ (ctx.config ?? {}) + const telemetry = config.telemetry + const slice = telemetry && typeof telemetry === 'object' && !Array.isArray(telemetry) + ? /** @type {Record} */ (telemetry) + : {} + + let host = DEFAULT_HOST + const hostRaw = slice.listen_host + if (typeof hostRaw === 'string' && hostRaw.length > 0) host = hostRaw + else if (hostRaw !== undefined) { + ctx.log.warn('claude.telemetry.config_invalid', { + [Attr.PLUGIN]: PLUGIN_NAME, + key: 'telemetry.listen_host', + value_type: typeof hostRaw, + }) + } + + let port = DEFAULT_TELEMETRY_PORT + let portConfigured = false + const portRaw = slice.listen_port + if (typeof portRaw === 'number' && Number.isInteger(portRaw) && portRaw >= 0 && portRaw <= 65535) { + port = portRaw + portConfigured = true + } else if (portRaw !== undefined) { + ctx.log.warn('claude.telemetry.config_invalid', { + [Attr.PLUGIN]: PLUGIN_NAME, + key: 'telemetry.listen_port', + value_type: typeof portRaw, + }) + } + + return { host, port, portConfigured } +} diff --git a/hypaware-core/plugins-workspace/claude/src/telemetry/spool.js b/hypaware-core/plugins-workspace/claude/src/telemetry/spool.js new file mode 100644 index 00000000..781b4b64 --- /dev/null +++ b/hypaware-core/plugins-workspace/claude/src/telemetry/spool.js @@ -0,0 +1,150 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' + +import { captureSpoolRoot } from '../../../../../src/core/capture_spool.js' + +/** + * Where Claude Code drops the raw request/response bodies attach asks it for. + * + * The path is fixed rather than configurable: attach writes it into the client + * settings, the listener reads it, and `hyp purge` and detach sweep it, so + * three unrelated surfaces have to agree on it without being told. The parent + * is core's capture-spool root rather than a second spelling of it, which is + * what makes this directory one `hyp purge` empties and one detach is allowed + * to sweep. + * + * @ref LLP 0253#spool-location [implements]: `/spool/claude-bodies`, + * owner-only + */ + +/** This client's directory name under the shared capture-spool root. */ +const SPOOL_DIRNAME = 'claude-bodies' + +/** + * Default byte cap for the spool. + * + * @ref LLP 0253#byte-cap [implements]: bounded by a configured cap, default + * 512 MB, so a down daemon can never fill the disk + */ +export const DEFAULT_SPOOL_MAX_BYTES = 512 * 1024 * 1024 + +/** + * The body spool directory for a HypAware home. + * + * @param {string} hypHome + * @returns {string} + */ +export function claudeBodySpoolDir(hypHome) { + return path.join(captureSpoolRoot(hypHome), SPOOL_DIRNAME) +} + +/** + * Create the spool directory owner-only, before anything is told to write into + * it. `mkdir`'s mode is filtered by the process umask, so the permission is + * set explicitly afterwards rather than hoped for; an existing directory is + * tightened the same way, which is what repairs a directory Claude Code + * created itself at the default mode. + * + * @ref LLP 0253#spool-location [implements]: created mode 0700, because raw + * prompts must not be world-readable + * @param {string} dir + * @returns {Promise} the same directory, for chaining + */ +export async function ensureClaudeBodySpool(dir) { + await fs.mkdir(dir, { recursive: true, mode: 0o700 }) + await fs.chmod(dir, 0o700) + return dir +} + +/** + * Tighten the spool's permissions IF it already exists, and create nothing. + * + * The daemon's job on this directory is repair, not creation: attach is what + * brings the spool into being (it is the same write that tells Claude Code + * where to put bodies), so a daemon that creates it anyway announces a capture + * surface on a machine that never attached this client - and does it against + * whatever `HYP_HOME` the activation context happened to resolve, which is how + * a test run reaches the developer's real `~/.hyp`. A missing directory is the + * normal state for an unattached install, so it is silently nothing to do. + * + * @ref LLP 0253#spool-location [implements]: the daemon keeps the directory + * owner-only; it is not the thing that mints it + * @param {string} dir + * @returns {Promise} whether a directory was found and tightened + */ +export async function tightenClaudeBodySpool(dir) { + try { + const stat = await fs.stat(dir) + if (!stat.isDirectory()) return false + } catch { + return false + } + await fs.chmod(dir, 0o700) + return true +} + +/** + * Enforce the spool's byte cap: when the directory's regular files sum + * past `maxBytes`, delete files strictly oldest-first (mtime, then name + * for a stable order when mtimes tie) until the total fits. + * + * The eviction direction is settled, not incidental: the newest bodies + * are the ones whose events are still arriving, so they are the ones + * worth keeping. An evicted body is not lost content: the transcript + * backfill path recovers the session later. + * + * Runs concurrently with Claude Code writing new files and with the + * listener deleting projected ones, so every per-file stat and unlink + * tolerates the file vanishing underneath it. + * + * @ref LLP 0253#byte-cap [implements]: oldest files are removed first when the + * cap is exceeded, enforced by the daemon rather than by hoping the reader + * keeps up + * @param {string} dir + * @param {number} maxBytes + * @returns {Promise<{ spoolBytes: number, evictedCount: number, evictedBytes: number }>} + */ +export async function enforceClaudeBodySpoolCap(dir, maxBytes) { + /** @type {Array<{ name: string, size: number, mtimeMs: number }>} */ + const files = [] + /** @type {import('node:fs').Dirent[]} */ + let entries + try { + entries = await fs.readdir(dir, { withFileTypes: true }) + } catch (err) { + if (/** @type {NodeJS.ErrnoException} */ (err).code === 'ENOENT') { + return { spoolBytes: 0, evictedCount: 0, evictedBytes: 0 } + } + throw err + } + for (const entry of entries) { + if (!entry.isFile()) continue + try { + const stat = await fs.stat(path.join(dir, entry.name)) + files.push({ name: entry.name, size: stat.size, mtimeMs: stat.mtimeMs }) + } catch { + // Deleted between readdir and stat: it no longer counts. + } + } + + let spoolBytes = files.reduce((sum, f) => sum + f.size, 0) + let evictedCount = 0 + let evictedBytes = 0 + if (spoolBytes <= maxBytes) return { spoolBytes, evictedCount, evictedBytes } + + files.sort((a, b) => a.mtimeMs - b.mtimeMs || (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)) + for (const file of files) { + if (spoolBytes <= maxBytes) break + try { + await fs.rm(path.join(dir, file.name), { force: true }) + } catch { + continue + } + spoolBytes -= file.size + evictedCount += 1 + evictedBytes += file.size + } + return { spoolBytes, evictedCount, evictedBytes } +} diff --git a/hypaware-core/plugins-workspace/claude/src/types.d.ts b/hypaware-core/plugins-workspace/claude/src/types.d.ts index ecbe9ddb..9157e2fb 100644 --- a/hypaware-core/plugins-workspace/claude/src/types.d.ts +++ b/hypaware-core/plugins-workspace/claude/src/types.d.ts @@ -14,6 +14,127 @@ export interface SessionContextRecord { ts: string | undefined } +/** + * One decoded Claude Code telemetry event: an OTLP log record with its + * `AnyValue` wrappers removed, keyed by the `event.name` attribute + * rather than by the record body. + */ +export interface ClaudeTelemetryEvent { + /** `event.name`, e.g. `user_prompt`, `assistant_response`, `api_request`. */ + name: string + /** `event.timestamp` (ISO-8601), falling back to the record's `timeUnixNano`. */ + timestamp?: string + /** `event.sequence`, Claude Code's per-session ordering counter. */ + sequence?: number + /** Every attribute on the record, unwrapped. */ + attributes: Record +} + +/** + * Session-level identity repeated on every Claude Code event. Collected + * once per batch so the projection carries it without each message + * restating it. + */ +export interface ClaudeTelemetrySessionFacts { + clientVersion?: string + entrypoint?: string + userId?: string + organizationId?: string + terminalType?: string + querySource?: string + agentName?: string + model?: string + startedAt?: string + /** + * Exchange-level fields only a spooled request body carries (events never + * do): the system prompt and the tool declarations, stamped on every row of + * the session's projection. + */ + systemText?: string + tools?: unknown +} + +/** + * What the usage policy says about one session at ingest, resolved from the + * cwd its SessionStart hook recorded (LLP 0254 #policy-inline). + * + * `class` is a `UsageClass` plus `'undetermined'`, the state a session is in + * while (or because) no hook record names its cwd: not `full`, because nothing + * was asked, and not `ignore`, because nothing said so. + */ +export interface ClaudeTelemetrySessionVerdict { + class: 'ignore' | 'local-only' | 'full' | 'undetermined' + /** The cwd the verdict was resolved from; absent when undetermined. */ + cwd?: string + /** Absolute path of the governing `.hypignore` or machine-local list. */ + governedBy?: string | null + /** The raw token before the fail-safe clamp. */ + declared?: string | null + /** Present only on a fail-safe clamp of an unknown token. */ + warn?: string +} + +/** + * What one received batch suppressed, accumulated across the opt-out gate and + * the usage-policy gate so the batch's span reports one total per outcome + * rather than whichever gate ran last. + */ +export interface BatchSuppressionTally { + /** Events a policy answered "no" for: the opt-out, or an `ignore` cwd. */ + eventsDropped: number + /** Events withheld because no verdict existed yet (no hook record). */ + eventsUndetermined: number + /** Spooled bodies deleted unread across both. */ + bodiesDropped: number +} + +/** + * Mutable counters one running Claude telemetry listener accumulates, + * surfaced through `status()` details. + */ +export interface ClaudeTelemetryListenerState { + rowsWritten: number + rowsSkipped: number + /** Rows written to `claude_telemetry_events`, counted apart from the message rows. */ + telemetryRowsWritten: number + eventsReceived: number + /** + * Events suppressed at ingest by a policy that said no: the per-session + * opt-out (LLP 0256) or an `ignore` cwd (LLP 0254 #policy-inline). + */ + eventsDropped: number + /** + * Events withheld at ingest because the session's cwd was not known, so no + * policy verdict existed to write under (LLP 0257 S10). + */ + eventsUndetermined: number + lastEventAt: string | undefined + lastError: string | undefined + listenFallbackFrom: number | undefined + /** Spool size as of the last sweep, decremented as bodies are consumed. */ + spoolBytes: number + bodiesProjected: number + bodiesDeleted: number + /** Bodies deleted unread because their session was policy-dropped. */ + bodiesDropped: number + bodiesEvicted: number + bodiesMissing: number + bodiesUnparseable: number +} + +/** + * One raw body file Claude Code dropped into the spool, located through an + * `api_request_body` / `api_response_body` event's `body_ref` and parsed. A + * `request` is a full Anthropic Messages request (system, tools, message + * history); a `response` is the assistant message the API returned. + */ +export interface SpooledClaudeBody { + kind: 'request' | 'response' + /** Resolved absolute path, proven to live inside the spool directory. */ + file: string + body: Record +} + export interface TranscriptEntry { sessionId: string role: string | undefined @@ -98,8 +219,12 @@ export interface ClaudeAttachOptions { * `ANTHROPIC_BASE_URL` at the local gateway. Defaults to `base_url` so a * caller that has not been taught about proxy mode cannot acquire it by * accident. See LLP 0231. + * + * `otel` writes neither routing key: it turns on Claude Code's own telemetry + * export, so the client talks to Anthropic directly and reports to the local + * listener. See LLP 0258. */ - mode?: 'proxy' | 'base_url' + mode?: 'proxy' | 'base_url' | 'otel' /** * Absolute path to the machine-local CA certificate. Required in `proxy` * mode, and its existence is the preflight: the gateway writes it only once @@ -107,12 +232,44 @@ export interface ClaudeAttachOptions { * Code's HTTPS rather than just its capture. */ caCertPath?: string + /** + * Port of the Claude telemetry listener, written into + * `env.OTEL_EXPORTER_OTLP_ENDPOINT`. Required in `otel` mode, and distinct + * from `port`: that one stays the gateway's, because it is what the + * attach-drift check compares against. + */ + telemetryPort?: number + /** + * Absolute path to the raw body spool, written into + * `env.OTEL_LOG_RAW_API_BODIES` and recorded on the marker so detach and + * purge can sweep it. Required in `otel` mode. + */ + spoolDir?: string + /** + * The installed Claude Code version, when it could be read. `otel` mode + * refuses below the floor (LLP 0258 #version-floor); an undetectable + * version is not a refusal, so `undefined` proceeds. + */ + claudeVersion?: string } export interface ClaudeAttachChanged { changed: true - /** The pre-existing `env.ANTHROPIC_BASE_URL` attach backed up, if any. */ + /** + * The pre-existing value of the env key this mode took over + * (`ANTHROPIC_BASE_URL`, `HTTPS_PROXY`, or `OTEL_EXPORTER_OTLP_ENDPOINT`), + * if any. A display copy: userinfo-redacted except in `base_url` mode. + */ prevValue?: string + /** + * The mode the prior `_hypaware` marker recorded, when one of the three + * known modes. `proxy` is the one the caller acts on: a proxy attach left + * residue outside the settings file (the launchd environment, the keychain + * trust) that the mode switch alone cannot reach, and by the time the caller + * runs, this write has already replaced the marker that said so. Absent on a + * first attach and on legacy markers that predate modes. + */ + priorMode?: 'proxy' | 'base_url' | 'otel' /** * One notice per `env` / `hooks` block this run found present on disk with * the wrong JSON type and had to rebuild. Attach backs the displaced value diff --git a/hypaware-core/plugins-workspace/codex/skills/hypaware-reference/SKILL.md b/hypaware-core/plugins-workspace/codex/skills/hypaware-reference/SKILL.md index bc3bbfc6..3a247644 100644 --- a/hypaware-core/plugins-workspace/codex/skills/hypaware-reference/SKILL.md +++ b/hypaware-core/plugins-workspace/codex/skills/hypaware-reference/SKILL.md @@ -86,11 +86,12 @@ curated HypAware registry. with no repo breadcrumb. - Stop recording *this conversation* - `hyp session ignore` drops this session's - exchanges at the gateway; `hyp session unignore` resumes, and `hyp session - status` reports which it is right now. Each resolves the session id itself - (Claude and Codex) and fails closed rather than guessing. The opt-out is - in-memory: a gateway restart drops it, and a fork (`claude --fork-session`, - `codex fork`) mints a new id it no longer covers. + exchanges at every local recorder (the gateway, and the Claude telemetry + listener when one is running); `hyp session unignore` resumes, and + `hyp session status` reports which it is right now. Each resolves the + session id itself (Claude and Codex) and fails closed rather than guessing. + The opt-out is in-memory: a daemon restart drops it, and a fork + (`claude --fork-session`, `codex fork`) mints a new id it no longer covers. - Decide what happens in new folders - by default they sync with no question; `hyp policy folders ask` asks once per new folder instead, and `hyp policy folders sync` returns to the default. It gates the question diff --git a/hypaware-core/plugins-workspace/otel/src/collector.js b/hypaware-core/plugins-workspace/otel/src/collector.js index 9c4e327b..95df1ec3 100644 --- a/hypaware-core/plugins-workspace/otel/src/collector.js +++ b/hypaware-core/plugins-workspace/otel/src/collector.js @@ -12,7 +12,7 @@ import { flattenOtlpMetrics } from './otlp/metrics.js' /** * @import { PluginActivationContext, PluginLogger } from '../../../../hypaware-plugin-kernel-types.js' - * @import { OtlpRequest } from './types.js' + * @import { OtlpRequest } from '../../../../src/core/otlp/types.js' */ const FLATTENERS = { diff --git a/hypaware-core/plugins-workspace/otel/src/source.js b/hypaware-core/plugins-workspace/otel/src/source.js index defce2f6..eceb650c 100644 --- a/hypaware-core/plugins-workspace/otel/src/source.js +++ b/hypaware-core/plugins-workspace/otel/src/source.js @@ -1,6 +1,6 @@ // @ts-check -import { createOtlpServer, listenAndResolve } from './server.js' +import { createOtlpJsonServer, listenAndResolve } from '../../../../src/core/otlp/server.js' import { makeReceiveHandler, stampBoundAddress } from './collector.js' /** @@ -10,6 +10,9 @@ import { makeReceiveHandler, stampBoundAddress } from './collector.js' const DEFAULT_HOST = '127.0.0.1' const DEFAULT_PORT = 4318 +/** What this listener calls itself on the wire banner and in bind errors. */ +const LISTENER_NAME = 'hypaware/otel' + /** * `startOtelSource` is the `SourceContribution.start` callback. It owns * the lifecycle of one HTTP listener: binds it, stamps `listen_host` / @@ -29,8 +32,9 @@ export async function startOtelSource(ctx) { /** @type {{ rowsWritten: number, lastError: string | undefined }} */ const state = { rowsWritten: 0, lastError: undefined } const handler = makeReceiveHandler(ctx, state, ctx.log) - const server = createOtlpServer({ handle: handler }) - const bound = await listenAndResolve(server, host, port) + // @ref LLP 0257#registration [implements]: the transport is shared core machinery; only the receive handler below is otel-owned + const server = createOtlpJsonServer({ name: LISTENER_NAME, handler: { handle: handler } }) + const bound = await listenAndResolve(server, host, port, LISTENER_NAME) stampBoundAddress(bound.host, bound.port) ctx.log.info('otel.listener_started', { listen_host: bound.host, diff --git a/hypaware-core/plugins-workspace/otel/src/types.d.ts b/hypaware-core/plugins-workspace/otel/src/types.d.ts deleted file mode 100644 index c9a18834..00000000 --- a/hypaware-core/plugins-workspace/otel/src/types.d.ts +++ /dev/null @@ -1,11 +0,0 @@ -export type OtlpSignal = 'logs' | 'traces' | 'metrics' - -export interface OtlpRequest { - signal: OtlpSignal - data: unknown - payloadBytes: number -} - -export interface OtlpReceiveHandler { - handle(req: OtlpRequest): Promise -} diff --git a/hypaware-core/smoke/flows/claude_attach_detach.js b/hypaware-core/smoke/flows/claude_attach_detach.js index 5127a052..a7ce31be 100644 --- a/hypaware-core/smoke/flows/claude_attach_detach.js +++ b/hypaware-core/smoke/flows/claude_attach_detach.js @@ -19,16 +19,23 @@ import { resolveDependencies } from '../../../src/core/dep_graph.js' import { requireAiGatewayRuntime } from '../../plugins-workspace/ai-gateway/src/runtime.js' /** - * Phase 8.4 smoke. Brings up `@hypaware/ai-gateway` + `@hypaware/claude` - * in a temp HYP_HOME with HOME pointed at the same tmp tree so the - * Claude settings file lives under it. Asserts the §Phase 8.4 contract - * from the implementation plan: + * Phase 8.4 smoke, updated for LLP 0258's `otel` attach mode. Brings up + * `@hypaware/ai-gateway` + `@hypaware/claude` in a temp HYP_HOME with + * HOME pointed at the same tmp tree so the Claude settings file lives + * under it. Asserts: * * - `hyp attach --client claude` patches `~/.claude/settings.json` - * with the HypAware marker, `env.ANTHROPIC_BASE_URL`, and the - * managed hook entries (golden compare): `session-context` on every - * managed event, plus the LLP 0106 `classify-cwd` hook, which the - * plugin scopes to the two fresh-cwd events. + * with the HypAware marker, the LLP 0258 telemetry `env` block + * (golden compare against the exact key set), and the managed hook + * entries: `session-context` on every managed event, plus the LLP + * 0106 `classify-cwd` hook, which the plugin scopes to the two + * fresh-cwd events. + * - No routing key is written: `ANTHROPIC_BASE_URL`, `HTTPS_PROXY`, + * and `NODE_EXTRA_CA_CERTS` all stay absent, which is the Remote + * Control predicate holding with no override keys (LLP 0258 + * #env-keys). + * - The marker records `mode: 'otel'` and the spool directory (LLP + * 0258 #marker-and-spool). * - A `client.attach` span exists with `hyp_plugin=@hypaware/claude`, * `client_name=claude`, `status=ok`, `restored=false`. * - `hyp detach --client claude` removes the managed keys and the @@ -36,6 +43,7 @@ import { requireAiGatewayRuntime } from '../../plugins-workspace/ai-gateway/src/ * - A `client.detach` span exists with `status=ok`, `restored=true`. * * @param {{ harness: any, expect: any }} args + * @ref LLP 0258#env-keys [tests]: the golden compare pins the exact env block attach writes */ export async function run({ harness, expect }) { const obs = installObservability() @@ -61,6 +69,12 @@ export async function run({ harness, expect }) { const previousHome = process.env.HOME process.env.HOME = fakeHome + // Pin the version the floor check sees: without this the smoke would + // inherit whatever `claude` binary the machine running it carries, and a + // stale install would flip the attach below to a refusal. + // @ref LLP 0258#version-floor [tests]: a version at or above the floor attaches + const previousClaudeVersion = process.env.HYP_CLAUDE_CODE_VERSION + process.env.HYP_CLAUDE_CODE_VERSION = '2.1.233' try { const registry = createCommandRegistry() @@ -170,11 +184,61 @@ export async function run({ harness, expect }) { attached?.permissions?.allow, (v) => Array.isArray(v) && v.length === 1 && v[0] === 'Bash(ls *)' ) + // The LLP 0258 #env-keys golden compare: the exact telemetry block, and + // only it. Each flag is asserted by value so a silently renamed or + // dropped key fails here instead of as an empty dataset in production. expect.that( - 'settings: env.ANTHROPIC_BASE_URL points at the local gateway', - attached?.env?.ANTHROPIC_BASE_URL, + 'settings: env.CLAUDE_CODE_ENABLE_TELEMETRY is on', + attached?.env?.CLAUDE_CODE_ENABLE_TELEMETRY, + (v) => v === '1' + ) + expect.that( + 'settings: both exporters are otlp', + [attached?.env?.OTEL_LOGS_EXPORTER, attached?.env?.OTEL_METRICS_EXPORTER], + (v) => v[0] === 'otlp' && v[1] === 'otlp' + ) + expect.that( + 'settings: the exporter protocol is http/json', + attached?.env?.OTEL_EXPORTER_OTLP_PROTOCOL, + (v) => v === 'http/json' + ) + expect.that( + 'settings: env.OTEL_EXPORTER_OTLP_ENDPOINT points at the loopback listener', + attached?.env?.OTEL_EXPORTER_OTLP_ENDPOINT, (v) => typeof v === 'string' && /^http:\/\/127\.0\.0\.1:\d+$/.test(v) ) + expect.that( + 'settings: all three content flags are on', + [ + attached?.env?.OTEL_LOG_USER_PROMPTS, + attached?.env?.OTEL_LOG_ASSISTANT_RESPONSES, + attached?.env?.OTEL_LOG_TOOL_DETAILS, + ], + (v) => v.every((flag) => flag === '1') + ) + expect.that( + 'settings: env.OTEL_LOG_RAW_API_BODIES names the spool under HYP_HOME', + attached?.env?.OTEL_LOG_RAW_API_BODIES, + (v) => + typeof v === 'string' && + v.startsWith('file:') && + v.endsWith(path.join('spool', 'claude-bodies')) && + v.includes(harness.hypHome) + ) + // The Remote Control predicate, stated as absences: no base URL change, + // no proxy keys. This is what lets Claude Code keep treating the + // endpoint as first party with no override keys at all. + // @ref LLP 0258#env-keys [tests]: ANTHROPIC_BASE_URL, HTTPS_PROXY, and NODE_EXTRA_CA_CERTS are not written + expect.that( + 'settings: no routing key was written (base URL, proxy, CA all absent)', + attached?.env, + (v) => + v !== null && + typeof v === 'object' && + !Object.hasOwn(v, 'ANTHROPIC_BASE_URL') && + !Object.hasOwn(v, 'HTTPS_PROXY') && + !Object.hasOwn(v, 'NODE_EXTRA_CA_CERTS') + ) expect.that( 'settings: _hypaware marker has the recorded port, version, and state file', attached?._hypaware, @@ -187,6 +251,37 @@ export async function run({ harness, expect }) { typeof v.state_file === 'string' && v.state_file.endsWith('session-context.jsonl') ) + // @ref LLP 0258#marker-and-spool [tests]: the marker records the mode and the spool directory detach and purge sweep + expect.that( + 'settings: marker records mode=otel and the spool directory', + attached?._hypaware, + (v) => + v !== null && + typeof v === 'object' && + v.mode === 'otel' && + typeof v.spool_dir === 'string' && + path.isAbsolute(v.spool_dir) && + v.spool_dir.endsWith(path.join('spool', 'claude-bodies')) + ) + expect.that( + 'settings: the marker manages exactly the nine telemetry keys', + attached?._hypaware?.managed?.env, + (v) => + v !== null && + typeof v === 'object' && + Object.keys(v).length === 9 && + Object.hasOwn(v, 'CLAUDE_CODE_ENABLE_TELEMETRY') && + Object.hasOwn(v, 'OTEL_LOG_RAW_API_BODIES') + ) + // The spool exists, owner-only, before any session is told to write into + // it. + // @ref LLP 0253#spool-location [tests]: created mode 0700 under HYP_HOME + const spoolStat = await fs.stat(path.join(harness.hypHome, 'spool', 'claude-bodies')) + expect.that( + 'spool: directory created owner-only at attach', + spoolStat, + (v) => v.isDirectory() && (v.mode & 0o777) === 0o700 + ) // LLP 0106 settles that attach installs the classification hook *alongside* // the existing session-context hook, which is what makes a golden compare // expecting session-context on its own stale. @@ -326,6 +421,8 @@ export async function run({ harness, expect }) { } finally { if (previousHome === undefined) delete process.env.HOME else process.env.HOME = previousHome + if (previousClaudeVersion === undefined) delete process.env.HYP_CLAUDE_CODE_VERSION + else process.env.HYP_CLAUDE_CODE_VERSION = previousClaudeVersion } } diff --git a/hypaware-core/smoke/flows/claude_telemetry_capture.js b/hypaware-core/smoke/flows/claude_telemetry_capture.js new file mode 100644 index 00000000..4ed5859e --- /dev/null +++ b/hypaware-core/smoke/flows/claude_telemetry_capture.js @@ -0,0 +1,1244 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' +import process from 'node:process' +import { Readable } from 'node:stream' + +import { Attr, installObservability, runRoot } from '../../../src/core/observability/index.js' +import { dispatch } from '../../../src/core/cli/dispatch.js' +import { createCommandRegistry } from '../../../src/core/registry/commands.js' +import { registerCoreCommands } from '../../../src/core/cli/core_commands.js' +import { createKernelRuntime } from '../../../src/core/runtime/activation.js' +import { activatePlugins } from '../../../src/core/runtime/loader.js' +import { loadManifests } from '../../../src/core/manifest.js' +import { resolveDependencies } from '../../../src/core/dep_graph.js' +import { claudeBodySpoolDir } from '../../plugins-workspace/claude/src/telemetry/spool.js' + +/** + * The Claude telemetry listener, end to end in a temp HypAware home. + * + * Activates `@hypaware/ai-gateway` + `@hypaware/claude`, drives the + * SessionStart hook, starts the listener source on a dynamic port, and + * POSTs a real-shaped Claude Code OTLP/JSON batch at it (`user_prompt`, + * `api_request`, `assistant_response`, the two body events pointing at + * spooled body files, plus the behavioral events - `tool_decision`, + * `permission_mode_changed`, `tool_result`, and the hook pair). Then + * asserts, through `hyp query sql`: + * + * - the rows landed in `ai_gateway_messages` with native uuid identity, + * the prompt and response text, the model, the usage the + * `api_request` event carried, and the cwd the hook recorded; + * - the behavioral events landed in `claude_telemetry_events`, one row + * per event with the hot fields typed (name, session, tool, decision, + * source, cost) and the rest preserved in the attributes JSON, with + * the content events NOT among them; a metrics POST lands its data + * points in the same dataset; the dataset enumerates alongside + * `ai_gateway_messages` and carries its central-forwarding signal; + * - the spooled bodies filled the gaps events never carry: system_text + * and the tools list on every row, the untruncated tool args, the + * tool result, and the thinking signature, and both body files are + * DELETED once projected; + * - the spool is owner-only and its byte cap evicts oldest-first at + * startup (a pre-staged over-cap body is gone before the first POST); + * - a session whose body was evicted still completes: its events land, + * and transcript backfill recovers the tool content the body held; + * - a replay of the same batch adds nothing; + * - a SECOND producer over the same session (transcript backfill, + * whose transcript carries the same uuids) adds nothing either, so + * the migration overlap window is harmless; + * - a non-JSON content type is refused the way the OTLP receiver + * refuses it, without disturbing the rows; + * - the capture spans say the intended path ran. + * + * @param {{ harness: any, expect: any }} args + * @ref LLP 0257#testing [tests]: the primary seam is a hermetic smoke, content + * in at the HTTP endpoint, body fixtures in the spool, rows out of + * `hyp query sql` + */ +export async function run({ harness, expect }) { + const obs = installObservability() + if (!obs.tracer.provider) { + throw new Error( + 'claude_telemetry_capture: tracer provider not installed - expected HYP_DEV_TELEMETRY=1' + ) + } + + const sessionId = `otel-${harness.devRunId}` + const userUuid = `u-user-${harness.devRunId}` + const assistantUuid = `u-asst-${harness.devRunId}` + const requestId = `req_${harness.devRunId}` + const promptText = 'Run ls, then read notes.txt.' + const responseText = 'This is a spike repo.' + const systemText = 'You are Claude Code, operating inside the smoke.' + const tools = [{ name: 'Read', description: 'Read a file', input_schema: { type: 'object' } }] + // Long enough that the event-side 512-char clip would have truncated it: + // only the spooled body carries it whole. + const longToolArg = 'n'.repeat(600) + const toolResultText = 'notes.txt: spike findings' + const thinkingText = 'The file confirms this is a spike repo.' + const thinkingSignature = `sig-${harness.devRunId}` + + // The evicted-session cast: its body is pre-staged over the cap and + // swept away at listener start, before its events ever arrive. + const session2 = `otel2-${harness.devRunId}` + const user2Uuid = `u2-user-${harness.devRunId}` + const toolAsst2Uuid = `u2-tool-${harness.devRunId}` + const toolResult2Uuid = `u2-result-${harness.devRunId}` + const assistant2Uuid = `u2-asst-${harness.devRunId}` + const request2Id = `req2_${harness.devRunId}` + const prompt2Text = 'What files are here?' + const response2Text = 'Just a README.' + + const cacheRoot = path.join(harness.stateDir, 'cache') + const registry = createCommandRegistry() + registerCoreCommands(registry) + const kernel = createKernelRuntime({ commandRegistry: registry, cacheRoot }) + + const pluginsRoot = path.resolve(import.meta.dirname, '..', '..', 'plugins-workspace') + const pluginDirs = [ + path.join(pluginsRoot, 'ai-gateway'), + path.join(pluginsRoot, 'claude'), + ] + + // The SECOND producer for the overlap assertion: a transcript whose + // uuids are the ones the events carry. `hyp backfill claude` reads it + // from `/.claude/projects`, and the Claude plugin captures HOME + // at activation, so it is staged first. + const fakeHome = path.join(harness.tmpDir, 'home') + const projectsDir = path.join(fakeHome, '.claude', 'projects', 'some-repo') + await fs.mkdir(projectsDir, { recursive: true }) + await fs.writeFile( + path.join(projectsDir, `${sessionId}.jsonl`), + [ + JSON.stringify({ + sessionId, + uuid: userUuid, + parentUuid: null, + type: 'user', + message: { role: 'user', content: promptText }, + timestamp: '2026-08-17T19:30:24.450Z', + }), + JSON.stringify({ + sessionId, + uuid: assistantUuid, + parentUuid: userUuid, + type: 'assistant', + message: { role: 'assistant', content: [{ type: 'text', text: responseText }] }, + timestamp: '2026-08-17T19:30:31.009Z', + }), + ].join('\n') + '\n', + 'utf8' + ) + + // The body spool, as Claude Code would leave it: created by the client at + // the default mode (the listener must tighten it), already holding one + // body larger than the configured cap (the startup sweep must evict it). + // @ref LLP 0253#byte-cap [tests]: the cap is config, eviction is oldest-first + const spoolDir = claudeBodySpoolDir(harness.hypHome) + await fs.mkdir(spoolDir, { recursive: true, mode: 0o755 }) + await fs.chmod(spoolDir, 0o755) + const evictedBodyPath = path.join(spoolDir, `${session2}-req.json`) + await fs.writeFile(evictedBodyPath, JSON.stringify({ + model: 'claude-haiku-4-5-20251001', + system: [{ type: 'text', text: systemText }], + tools, + messages: [ + { role: 'user', content: prompt2Text }, + { + role: 'assistant', + content: [ + { type: 'tool_use', id: 'toolu_evicted', name: 'Bash', input: { command: `ls ${longToolArg}` } }, + ], + }, + { + role: 'user', + content: [{ type: 'tool_result', tool_use_id: 'toolu_evicted', content: 'README.md' }], + }, + ], + }), 'utf8') + const spoolCapBytes = 256 + + const previousHome = process.env.HOME + process.env.HOME = fakeHome + + try { + await runRoot( + 'kernel.boot', + { + [Attr.COMPONENT]: 'kernel', + [Attr.OPERATION]: 'boot', + [Attr.SMOKE_NAME]: harness.smokeName, + [Attr.SMOKE_STEP]: 'claude_telemetry_activate', + [Attr.DEV_RUN_ID]: harness.devRunId, + status: 'ok', + }, + async () => { + const { loaded } = await loadManifests(pluginDirs) + if (loaded.length !== pluginDirs.length) { + throw new Error(`claude_telemetry_capture: expected ${pluginDirs.length} manifests, got ${loaded.length}`) + } + const resolution = await resolveDependencies(loaded.map((l) => l.manifest)) + if (resolution.unsatisfied.length > 0) { + throw new Error( + `claude_telemetry_capture: unsatisfied requirements: ${ + resolution.unsatisfied.map((u) => `${u.plugin}:${u.errorKind}`).join(', ') + }` + ) + } + const byName = new Map(loaded.map((l) => [l.manifest.name, l])) + const entries = resolution.order + .map((name) => byName.get(name)) + .filter((l) => l !== undefined) + .map((l) => ({ + manifest: l.manifest, + rootDir: l.rootDir, + // Port 0: the smoke reads the bound port back off the source + // status, the same way `hyp attach claude` will. + config: /** @type {any} */ (l.manifest.name === '@hypaware/claude' + ? { telemetry: { listen_host: '127.0.0.1', listen_port: 0, spool_max_bytes: spoolCapBytes } } + : {}), + })) + return activatePlugins({ + plugins: entries, + stateRoot: harness.stateDir, + runId: harness.devRunId, + runtime: kernel, + tmpRoot: path.join(harness.tmpDir, 'plugin-temp'), + }) + } + ) + + const env = { ...process.env, HYP_HOME: harness.hypHome } + + // ----- SessionStart hook: the source of cwd and git identity ----- + const stateFile = path.join( + harness.stateDir, 'plugins', '@hypaware/claude', 'session-context.jsonl' + ) + await fs.mkdir(path.dirname(stateFile), { recursive: true }) + const hookCode = await dispatch( + ['claude-hook', 'session-context', '--state-file', stateFile], + { + stdout: makeBuf(), + stderr: makeBuf(), + stdin: /** @type {any} */ (Readable.from([JSON.stringify({ + session_id: sessionId, + cwd: harness.tmpDir, + hook_event_name: 'SessionStart', + })])), + kernel, + registry, + env, + } + ) + expect.that('hook: session-context invocation exited 0', hookCode, (v) => v === 0) + const hook2Code = await dispatch( + ['claude-hook', 'session-context', '--state-file', stateFile], + { + stdout: makeBuf(), + stderr: makeBuf(), + stdin: /** @type {any} */ (Readable.from([JSON.stringify({ + session_id: session2, + cwd: harness.tmpDir, + hook_event_name: 'SessionStart', + })])), + kernel, + registry, + env, + } + ) + expect.that('hook: session-context invocation for the evicted session exited 0', hook2Code, (v) => v === 0) + + // ----- Start the listener ----- + const ctx = kernel.activationContexts.get('@hypaware/claude') + if (!ctx) throw new Error('claude_telemetry_capture: no activation context for @hypaware/claude') + await kernel.sources.start('claude-telemetry', ctx) + const started = kernel.sources.started('claude-telemetry') + if (!started) throw new Error('claude_telemetry_capture: source `claude-telemetry` not started') + const status = await /** @type {NonNullable} */ (started.status)() + const details = /** @type {{ listen_host?: string, listen_port?: number, spool_bytes?: number, bodies_evicted?: number }} */ (status.details ?? {}) + expect.that( + 'status: listener reports a loopback host and a bound port', + details, + (v) => v.listen_host === '127.0.0.1' && typeof v.listen_port === 'number' && v.listen_port > 0 + ) + const endpoint = `http://${details.listen_host}:${details.listen_port}` + + // ----- The spool after listener start: tightened, capped, swept ----- + const spoolMode = (await fs.stat(spoolDir)).mode & 0o777 + expect.that('spool: the listener tightened the client-created directory to owner-only', spoolMode, (v) => v === 0o700) + expect.that( + 'spool: the pre-staged over-cap body was evicted at startup', + await fileExists(evictedBodyPath), + (v) => v === false + ) + expect.that( + 'status: the eviction and the swept spool size are visible in the source status', + details, + (v) => v.bodies_evicted === 1 && v.spool_bytes === 0 + ) + + // ----- Session 1's body files, dropped the way Claude Code drops them ----- + const requestBodyPath = path.join(spoolDir, `${sessionId}-req.json`) + const responseBodyPath = path.join(spoolDir, `${sessionId}-resp.json`) + await fs.writeFile(requestBodyPath, JSON.stringify({ + model: 'claude-haiku-4-5-20251001', + system: [{ type: 'text', text: systemText }], + tools, + messages: [ + { role: 'user', content: promptText }, + { + role: 'assistant', + content: [ + { type: 'text', text: 'Reading notes.txt now.' }, + { type: 'tool_use', id: 'toolu_smoke', name: 'Read', input: { file_path: '/tmp/notes.txt', notes: longToolArg } }, + ], + }, + { + role: 'user', + content: [{ type: 'tool_result', tool_use_id: 'toolu_smoke', content: toolResultText }], + }, + ], + }), 'utf8') + await fs.writeFile(responseBodyPath, JSON.stringify({ + id: `msg_${harness.devRunId}`, + type: 'message', + role: 'assistant', + model: 'claude-haiku-4-5-20251001', + content: [ + { type: 'thinking', thinking: thinkingText, signature: thinkingSignature }, + { type: 'text', text: responseText }, + ], + stop_reason: 'end_turn', + usage: { input_tokens: 73, output_tokens: 113 }, + }), 'utf8') + + // ----- Refuse a non-JSON content type, like the OTLP receiver ----- + const badType = await fetch(`${endpoint}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'text/plain' }, + body: 'not otlp', + }) + expect.that('listener: a non-json content type is refused with 415', badType.status, (v) => v === 415) + await badType.text() + + // ----- POST one real-shaped batch ----- + const payload = buildTelemetryBatch({ + sessionId, + userUuid, + assistantUuid, + requestId, + promptText, + responseText, + requestBodyPath, + responseBodyPath, + }) + const posted = await postJson(`${endpoint}/v1/logs`, payload) + expect.that('listener: OTLP/JSON POST returned 200', posted.status, (v) => v === 200) + + // Projected, then deleted: the batch's write succeeded, so the raw + // bodies must be gone from disk. + // @ref LLP 0252#project-then-delete [tests]: deletion is the normal end of + // a body's life + expect.that( + 'spool: the request body file was deleted after projection', + await fileExists(requestBodyPath), + (v) => v === false + ) + expect.that( + 'spool: the response body file was deleted after projection', + await fileExists(responseBodyPath), + (v) => v === false + ) + + const sqlFor = (/** @type {string} */ session) => ` + select + role, + part_type, + content_text, + message_id, + part_id, + provider_uuid, + model, + cwd, + client_name, + conversation_source, + provider, + entrypoint, + parent_uuid, + permission_mode, + request_id, + system_text, + tool_name, + tool_call_id, + tool_result_for, + thinking_signature, + JSON_VALUE(tools, '$[0].name') as tool0_name, + JSON_VALUE(tool_args, '$.notes') as tool_arg_notes, + JSON_VALUE(tool_args, '$.command') as tool_arg_command, + JSON_VALUE(raw_frame, '$.type') as frame_type, + JSON_VALUE(raw_frame, '$.body_file') as frame_body_file, + raw_frame, + JSON_VALUE(attributes, '$.usage.output_tokens') as output_tokens, + JSON_VALUE(attributes, '$.usage.cache_read_tokens') as cache_read_tokens, + JSON_VALUE(attributes, '$.gateway.source') as producer, + JSON_VALUE(attributes, '$.claude.query_source') as query_source + from ai_gateway_messages + where session_id = '${session}' + order by message_index, part_index + `.trim().replace(/\s+/g, ' ') + const sql = sqlFor(sessionId) + + const rows = await queryRows({ sql, kernel, registry, env, expect, label: 'after the first batch' }) + expect.that( + 'query: the turn landed as five rows (prompt, tool_use, tool_result, thinking, response)', + rows, + (v) => Array.isArray(v) && v.length === 5, + ) + expect.that( + 'query: the rows follow the body\'s canonical message ordering', + rows.map((/** @type {any} */ r) => r.part_type), + (v) => JSON.stringify(v) === JSON.stringify(['text', 'tool_call', 'tool_result', 'reasoning', 'text']), + ) + + const user = rows.find((/** @type {any} */ r) => r.role === 'user' && r.part_type === 'text') + const assistant = rows.find((/** @type {any} */ r) => r.content_text === responseText) + expect.that( + 'query: the user row carries the native message uuid and the prompt text', + user, + (v) => v !== undefined && v.message_id === userUuid && v.provider_uuid === userUuid && v.content_text === promptText, + ) + expect.that( + 'query: the assistant row carries the native message uuid and the response text', + assistant, + (v) => v !== undefined && v.message_id === assistantUuid && v.content_text === responseText, + ) + expect.that( + 'query: the assistant row carries the model and the request id', + assistant, + (v) => v !== undefined && v.model === 'claude-haiku-4-5-20251001' && v.request_id === requestId, + ) + expect.that( + 'query: the api_request usage landed on the assistant row', + assistant, + (v) => v !== undefined && Number(v.output_tokens) === 113 && Number(v.cache_read_tokens) === 35212, + ) + + // ----- The gaps only the spooled bodies could fill ----- + // @ref LLP 0252#bodies-for-gaps [tests]: system text, tools, untruncated + // args, tool results, and thinking signatures come from the body files + const toolUse = rows.find((/** @type {any} */ r) => r.part_type === 'tool_call') + expect.that( + 'query: the tool_use row carries the FULL untruncated tool args from the body', + toolUse, + (v) => v !== undefined && v.tool_name === 'Read' && v.tool_call_id === 'toolu_smoke' && + v.tool_arg_notes === longToolArg, + ) + const toolResult = rows.find((/** @type {any} */ r) => r.part_type === 'tool_result') + expect.that( + 'query: the tool_result row carries the result the wire events never showed', + toolResult, + (v) => v !== undefined && v.role === 'user' && v.tool_result_for === 'toolu_smoke' && + v.content_text === toolResultText, + ) + const reasoning = rows.find((/** @type {any} */ r) => r.part_type === 'reasoning') + expect.that( + 'query: the thinking row carries the signature from the response body', + reasoning, + (v) => v !== undefined && v.thinking_signature === thinkingSignature && v.content_text === thinkingText, + ) + for (const gapRow of [toolUse, toolResult, reasoning]) { + const frame = JSON.stringify(gapRow?.raw_frame ?? '') + expect.that( + `query: the ${gapRow?.part_type} row's raw_frame is a body pointer, never content`, + gapRow, + (v) => v !== undefined && + (v.frame_type === 'api_request_body' || v.frame_type === 'api_response_body') && + typeof v.frame_body_file === 'string' && + !frame.includes(longToolArg.slice(0, 32)) && + !frame.includes(thinkingText), + ) + } + + for (const row of rows) { + expect.that( + `query: the ${row.part_type} ${row.role} row carries the cwd the hook recorded`, + row.cwd, + (v) => v === harness.tmpDir, + ) + expect.that( + `query: the ${row.part_type} ${row.role} row is attributed to the claude client over anthropic`, + row, + (v) => v.client_name === 'claude' && v.provider === 'anthropic' && v.conversation_source === 'claude_code', + ) + expect.that( + `query: the ${row.part_type} ${row.role} row records the OTEL producer and the query source`, + row, + (v) => v.producer === 'otel' && v.query_source === 'sdk', + ) + expect.that( + `query: the ${row.part_type} ${row.role} row leaves the transcript-only columns null`, + row, + (v) => (v.parent_uuid ?? null) === null && (v.permission_mode ?? null) === null, + ) + expect.that( + `query: the ${row.part_type} ${row.role} row carries the app entrypoint`, + row.entrypoint, + (v) => v === 'sdk-cli', + ) + // Stamped exchange-level from the request body, on every row, exactly + // where the proxy path puts them. + expect.that( + `query: the ${row.part_type} ${row.role} row carries the body's system text and tools`, + row, + (v) => v.system_text === systemText && v.tool0_name === 'Read', + ) + } + + // ----- The behavioral half: claude_telemetry_events ----- + // @ref LLP 0255#row-shape [tests]: one row per event, hot fields typed + // (name, session, tool, decision, source, cost), attributes JSON for + // the rest + const eventsSqlFor = (/** @type {string} */ session) => ` + select + event_name, + session_id, + tool_name, + decision, + source, + cost_usd, + event_timestamp, + JSON_VALUE(attributes, '$.from_mode') as from_mode, + JSON_VALUE(attributes, '$.to_mode') as to_mode, + JSON_VALUE(attributes, '$.hook_name') as hook_name, + JSON_VALUE(attributes, '$.success') as hook_success, + JSON_VALUE(attributes, '$.input_tokens') as input_tokens, + JSON_VALUE(attributes, '$.decision') as json_decision, + JSON_VALUE(attributes, '$.value') as metric_value, + JSON_VALUE(attributes, '$.unit') as metric_unit + from claude_telemetry_events + where session_id = '${session}' + order by event_timestamp, event_name + `.trim().replace(/\s+/g, ' ') + const eventsSql = eventsSqlFor(sessionId) + + const eventRows = await queryRows({ sql: eventsSql, kernel, registry, env, expect, label: 'behavioral events' }) + expect.that( + 'events: one row per behavioral event, content and body events excluded', + eventRows.map((/** @type {any} */ r) => r.event_name), + (v) => JSON.stringify(v) === JSON.stringify([ + 'permission_mode_changed', + 'tool_decision', + 'tool_result', + 'hook_execution_start', + 'hook_execution_complete', + 'api_request', + ]), + ) + for (const row of eventRows) { + expect.that( + `events: the ${row.event_name} row carries the session id and a timestamp`, + row, + (v) => v.session_id === sessionId && typeof v.event_timestamp === 'string' && v.event_timestamp.length > 0, + ) + } + const toolDecision = eventRows.find((/** @type {any} */ r) => r.event_name === 'tool_decision') + expect.that( + 'events: the tool_decision row types the tool, the decision, and its source', + toolDecision, + (v) => v !== undefined && v.tool_name === 'Read' && v.decision === 'reject' && v.source === 'user_reject', + ) + expect.that( + 'events: a promoted hot field leaves the attributes JSON', + toolDecision, + (v) => v !== undefined && (v.json_decision ?? null) === null, + ) + const modeChange = eventRows.find((/** @type {any} */ r) => r.event_name === 'permission_mode_changed') + expect.that( + 'events: the permission_mode_changed row keeps its unpromoted attributes in the JSON', + modeChange, + (v) => v !== undefined && v.from_mode === 'default' && v.to_mode === 'acceptEdits' && + (v.tool_name ?? null) === null && (v.decision ?? null) === null, + ) + const apiRequest = eventRows.find((/** @type {any} */ r) => r.event_name === 'api_request') + expect.that( + 'events: the api_request row types the cost and keeps the token counts in the JSON', + apiRequest, + (v) => v !== undefined && Math.abs(Number(v.cost_usd) - 0.0047732) < 1e-9 && + Number(v.input_tokens) === 73, + ) + const hookComplete = eventRows.find((/** @type {any} */ r) => r.event_name === 'hook_execution_complete') + expect.that( + 'events: the hook_execution_complete row carries the hook identity and outcome', + hookComplete, + (v) => v !== undefined && v.hook_name === 'hypaware-session-context' && v.hook_success === 'true', + ) + + // The registration surfaces: the dataset enumerates alongside the + // existing ones, and carries the ingest signal central forwarding + // needs so it never falls back to the dataset name. + // @ref LLP 0255#owned-by-claude [tests]: registration sets the source signal + const statusOut = makeBuf() + const statusCode = await dispatch( + ['query', 'status'], + { stdout: statusOut, stderr: makeBuf(), kernel, registry, env } + ) + expect.that('dispatch: query status exited 0', statusCode, (v) => v === 0) + expect.that( + 'enumeration: claude_telemetry_events is listed alongside ai_gateway_messages', + statusOut.text(), + (v) => v.includes('claude_telemetry_events (@hypaware/claude)') && + v.includes('ai_gateway_messages (@hypaware/ai-gateway)'), + ) + expect.that( + 'registration: the dataset declares the claude_telemetry source signal', + kernel.query.getDataset('claude_telemetry_events')?.sourceSignal, + (v) => v === 'claude_telemetry', + ) + + // ----- A replayed batch adds nothing ----- + // The body files are already gone, so on replay the content events + // dedupe by part_id and the body refs read as missing, not as errors. + const replay = await postJson(`${endpoint}/v1/logs`, payload) + expect.that('listener: the replayed POST returned 200', replay.status, (v) => v === 200) + const afterReplay = await queryRows({ sql, kernel, registry, env, expect, label: 'after the replay' }) + expect.that( + 'query: replaying the same events did not duplicate the rows', + afterReplay, + (v) => Array.isArray(v) && v.length === 5, + ) + // The behavioral dataset has no pre-write dedupe by design (single + // producer, one POST per batch; a retry only follows a failed write, + // which wrote nothing). A manually re-POSTed identical batch is the + // lost-success-response window: the rows double, byte-identical, and + // cache compaction's content-hash layer owns the collapse. + const eventsAfterReplay = await queryRows({ sql: eventsSql, kernel, registry, env, expect, label: 'behavioral events after the replay' }) + expect.that( + 'events: the replayed batch appended its behavioral rows again, byte-identical', + eventsAfterReplay, + (v) => Array.isArray(v) && v.length === 12, + ) + + // ----- A second producer over the same session adds nothing ----- + // The overlap window of the proxy-to-OTEL migration, in miniature: + // transcript backfill re-materializes the same parts and the + // `part_id` dedupe collapses them onto the rows already stored. + const backfillOut = makeBuf() + const backfillErr = makeBuf() + const backfillCode = await dispatch( + ['backfill', 'claude', '--since', '2000-01-01T00:00:00.000Z', '--json'], + { stdout: backfillOut, stderr: backfillErr, kernel, registry, env: { ...env, DEV_RUN_ID: `${harness.devRunId}-backfill` } } + ) + expect.that('dispatch: backfill claude exited 0', backfillCode, (v) => v === 0) + const backfillRun = JSON.parse(backfillOut.text()) + const claudeProvider = backfillRun.providers.find((/** @type {any} */ p) => p.provider === 'claude') + expect.that( + 'backfill: the transcript producer wrote ZERO new rows over the OTEL capture', + claudeProvider, + (v) => v !== undefined && v.status === 'ok' && v.rows_written === 0, + ) + const afterBackfill = await queryRows({ sql, kernel, registry, env, expect, label: 'after backfill' }) + expect.that( + 'query: two producers over one session still dedupe to one set of rows', + afterBackfill, + (v) => Array.isArray(v) && v.length === 5, + ) + + // ----- An evicted session still completes via transcript backfill ----- + // Its body was swept at startup, so the batch lands only what the + // events carry: the prompt and the response text, no tool content and + // no system text. + // @ref LLP 0253#eviction-degrades [tests]: eviction degrades to backfill, + // never to loss + const payload2 = buildEvictedSessionBatch({ + sessionId: session2, + userUuid: user2Uuid, + assistantUuid: assistant2Uuid, + requestId: request2Id, + promptText: prompt2Text, + responseText: response2Text, + requestBodyPath: evictedBodyPath, + }) + const posted2 = await postJson(`${endpoint}/v1/logs`, payload2) + expect.that('listener: the evicted session\'s POST returned 200', posted2.status, (v) => v === 200) + const sql2 = sqlFor(session2) + const evictedRows = await queryRows({ sql: sql2, kernel, registry, env, expect, label: 'evicted session, events only' }) + expect.that( + 'query: the evicted session landed its two content events and nothing else', + evictedRows, + (v) => Array.isArray(v) && v.length === 2 && + v.every((r) => r.part_type === 'text' && (r.system_text ?? null) === null), + ) + + // The recovery path: the transcript Claude Code wrote all along holds + // the tool content the evicted body held. Staged only now, so the + // earlier zero-rows backfill assertion stays meaningful. + await fs.writeFile( + path.join(projectsDir, `${session2}.jsonl`), + [ + JSON.stringify({ + sessionId: session2, + uuid: user2Uuid, + parentUuid: null, + type: 'user', + message: { role: 'user', content: prompt2Text }, + timestamp: '2026-08-17T19:40:01.000Z', + }), + JSON.stringify({ + sessionId: session2, + uuid: toolAsst2Uuid, + parentUuid: user2Uuid, + type: 'assistant', + message: { + role: 'assistant', + model: 'claude-haiku-4-5-20251001', + content: [{ type: 'tool_use', id: 'toolu_evicted', name: 'Bash', input: { command: `ls ${longToolArg}` } }], + }, + timestamp: '2026-08-17T19:40:02.000Z', + }), + JSON.stringify({ + sessionId: session2, + uuid: toolResult2Uuid, + parentUuid: toolAsst2Uuid, + type: 'user', + message: { + role: 'user', + content: [{ type: 'tool_result', tool_use_id: 'toolu_evicted', content: 'README.md' }], + }, + timestamp: '2026-08-17T19:40:03.000Z', + }), + JSON.stringify({ + sessionId: session2, + uuid: assistant2Uuid, + parentUuid: toolResult2Uuid, + type: 'assistant', + message: { role: 'assistant', model: 'claude-haiku-4-5-20251001', content: [{ type: 'text', text: response2Text }] }, + timestamp: '2026-08-17T19:40:04.000Z', + }), + ].join('\n') + '\n', + 'utf8' + ) + const recoverOut = makeBuf() + const recoverCode = await dispatch( + ['backfill', 'claude', '--since', '2000-01-01T00:00:00.000Z', '--json'], + { stdout: recoverOut, stderr: makeBuf(), kernel, registry, env: { ...env, DEV_RUN_ID: `${harness.devRunId}-recover` } } + ) + expect.that('dispatch: the recovery backfill exited 0', recoverCode, (v) => v === 0) + const recoverRun = JSON.parse(recoverOut.text()) + const recoverProvider = recoverRun.providers.find((/** @type {any} */ p) => p.provider === 'claude') + expect.that( + 'backfill: recovery wrote exactly the two rows the evicted body held', + recoverProvider, + (v) => v !== undefined && v.status === 'ok' && v.rows_written === 2, + ) + const recoveredRows = await queryRows({ sql: sql2, kernel, registry, env, expect, label: 'evicted session, recovered' }) + expect.that( + 'query: the evicted session completed to four rows', + recoveredRows, + (v) => Array.isArray(v) && v.length === 4, + ) + const recoveredTool = recoveredRows.find((/** @type {any} */ r) => r.part_type === 'tool_call') + expect.that( + 'query: the recovered tool_use row carries native identity and the full args', + recoveredTool, + (v) => v !== undefined && v.part_id === `${toolAsst2Uuid}#0` && + v.tool_arg_command === `ls ${longToolArg}` && v.producer === 'backfill', + ) + expect.that( + 'query: the event-captured rows of the evicted session were not disturbed', + recoveredRows.filter((/** @type {any} */ r) => r.producer === 'otel').length, + (v) => v === 2, + ) + + // ----- The metrics half of the exporter config ----- + // The same env block turns on OTEL_METRICS_EXPORTER, so Claude Code + // POSTs its activity counters at /v1/metrics; each data point lands + // as one behavioral row named by its metric. + const postedMetrics = await postJson(`${endpoint}/v1/metrics`, buildMetricsBatch({ sessionId })) + expect.that('listener: the metrics POST returned 200', postedMetrics.status, (v) => v === 200) + const withMetrics = await queryRows({ sql: eventsSql, kernel, registry, env, expect, label: 'behavioral events with metrics' }) + const costRow = withMetrics.find((/** @type {any} */ r) => r.event_name === 'claude_code.cost.usage') + expect.that( + 'events: the cost metric data point landed with its value, unit, and session', + costRow, + (v) => v !== undefined && Math.abs(Number(v.metric_value) - 0.0047732) < 1e-9 && + v.metric_unit === 'USD' && v.session_id === sessionId && + typeof v.event_timestamp === 'string' && v.event_timestamp.length > 0, + ) + const locRow = withMetrics.find((/** @type {any} */ r) => r.event_name === 'claude_code.lines_of_code.count') + expect.that( + 'events: the lines-of-code metric data point landed with its integer value', + locRow, + (v) => v !== undefined && Number(v.metric_value) === 42, + ) + + const finalStatus = await /** @type {NonNullable} */ (started.status)() + const finalDetails = /** @type {any} */ (finalStatus.details ?? {}) + expect.that( + 'status: the listener counted the two projected bodies and an empty spool', + finalDetails, + (v) => v.bodies_projected === 2 && v.spool_bytes === 0 && v.bodies_evicted === 1, + ) + // 6 from the first batch, 6 from its replay, 1 (api_request) from the + // evicted session, 2 metric data points. + expect.that( + 'status: the behavioral row count is reported apart from the message rows', + finalDetails, + (v) => v.telemetry_rows_written === 15, + ) + + await kernel.sources.stop('claude-telemetry') + await obs.shutdown() + + // ----- Capture telemetry ----- + const traces = await expect.traces() + + const startSpans = traces.filter( + (/** @type {any} */ t) => t.name === 'source.start' && t.attributes?.hyp_source === 'claude-telemetry' + ) + expect.that( + 'traces: exactly one source.start span for the claude telemetry listener', + startSpans, + (v) => Array.isArray(v) && v.length === 1, + ) + expect.that( + 'traces: source.start carries the bound address', + startSpans[0]?.attributes, + (v) => v !== undefined && v.listen_host === details.listen_host && v.listen_port === details.listen_port, + ) + expect.that( + 'traces: source.start is tagged with the owning plugin', + startSpans[0]?.attributes?.[Attr.PLUGIN], + (v) => v === '@hypaware/claude', + ) + + const receives = traces.filter( + (/** @type {any} */ t) => t.name === 'claude.telemetry.receive' + ) + expect.that( + 'traces: one claude.telemetry.receive span per accepted batch', + receives, + (v) => Array.isArray(v) && v.length === 4, + ) + expect.that( + 'traces: the first receive span reports ok, the events it saw, and the rows it wrote', + receives[0]?.attributes, + (v) => v !== undefined && + v.status === 'ok' && + v.signal === 'logs' && + Number(v.payload_bytes) > 0 && + Number(v.event_count) === 10 && + Number(v.session_count) === 1 && + Number(v.row_count) === 5 && + Number(v.telemetry_row_count) === 6, + ) + expect.that( + 'traces: the first receive span counted the two bodies it projected and deleted', + receives[0]?.attributes, + (v) => v !== undefined && Number(v.body_count) === 2 && + Number(v.bodies_projected) === 2 && Number(v.bodies_deleted) === 2, + ) + expect.that( + 'traces: the replay receive span wrote nothing', + receives[1]?.attributes, + (v) => v !== undefined && Number(v.row_count) === 0, + ) + expect.that( + 'traces: the evicted session\'s receive span wrote its event rows without a body', + receives[2]?.attributes, + (v) => v !== undefined && Number(v.row_count) === 2 && Number(v.body_count) === 0, + ) + expect.that( + 'traces: the metrics receive span wrote only behavioral rows', + receives[3]?.attributes, + (v) => v !== undefined && v.signal === 'metrics' && Number(v.event_count) === 2 && + Number(v.row_count) === 0 && Number(v.telemetry_row_count) === 2, + ) + + const cacheAppends = traces.filter( + (/** @type {any} */ t) => + t.name === 'cache.append' && t.attributes?.hyp_dataset === 'ai_gateway_messages' + ) + expect.that( + 'traces: at least one cache.append for ai_gateway_messages', + cacheAppends, + (v) => Array.isArray(v) && v.length >= 1, + ) + const eventAppends = traces.filter( + (/** @type {any} */ t) => + t.name === 'cache.append' && t.attributes?.hyp_dataset === 'claude_telemetry_events' + ) + expect.that( + 'traces: at least one cache.append for claude_telemetry_events', + eventAppends, + (v) => Array.isArray(v) && v.length >= 1, + ) + + const logs = await expect.logs() + const batchLogs = logs.filter((/** @type {any} */ l) => l.body === 'claude.telemetry.batch') + expect.that( + 'logs: the batch log reports the event, row, and body counts', + batchLogs[0]?.attributes, + (v) => v !== undefined && Number(v.event_count) === 10 && + Number(v.rows_written) === 5 && Number(v.telemetry_rows_written) === 6 && + Number(v.bodies_projected) === 2, + ) + expect.that( + 'logs: the evicted session\'s batch log counts its missing body', + batchLogs[2]?.attributes, + (v) => v !== undefined && Number(v.bodies_missing) === 1, + ) + const evictLogs = logs.filter((/** @type {any} */ l) => l.body === 'claude.telemetry.spool_evicted') + expect.that( + 'logs: the startup sweep logged the eviction with counts', + evictLogs[0]?.attributes, + (v) => v !== undefined && Number(v.evicted_count) === 1 && + Number(v.spool_max_bytes) === spoolCapBytes, + ) + } finally { + if (previousHome === undefined) delete process.env.HOME + else process.env.HOME = previousHome + } +} + +// --------------------------------------------------------------------- +// Fixture + helpers +// --------------------------------------------------------------------- + +/** + * One turn as Claude Code 2.1.233 exports it: `user_prompt` and + * `assistant_response` project into messages, the two body events' + * `body_ref`s point into the spool, and the behavioral events + * (`permission_mode_changed`, `tool_decision`, `tool_result`, the hook + * pair, and `api_request`, which also feeds usage onto the assistant + * row) land in `claude_telemetry_events`. + * + * @param {{ + * sessionId: string, + * userUuid: string, + * assistantUuid: string, + * requestId: string, + * promptText: string, + * responseText: string, + * requestBodyPath: string, + * responseBodyPath: string, + * }} args + */ +function buildTelemetryBatch(args) { + const common = { + 'session.id': args.sessionId, + 'app.version': '2.1.233', + 'app.entrypoint': 'sdk-cli', + 'organization.id': '2efcd21e-aea6-42c6-9eda-a6e997ddcde4', + 'user.account_uuid': 'c9f39145-595f-4b31-9c66-c5c658a80aed', + 'user.email': 'someone@example.com', + 'terminal.type': 'ghostty', + 'prompt.id': `p-${args.sessionId}`, + } + return { + resourceLogs: [ + { + resource: { + attributes: kv({ + 'service.name': 'claude-code', + 'service.version': '2.1.233', + 'os.type': 'darwin', + }), + }, + scopeLogs: [ + { + scope: { name: 'com.anthropic.claude_code.events', version: '2.1.233' }, + logRecords: [ + logRecord('permission_mode_changed', '2026-08-17T19:30:20.000Z', { + ...common, + from_mode: 'default', + to_mode: 'acceptEdits', + trigger: 'user', + }), + logRecord('user_prompt', '2026-08-17T19:30:24.450Z', { + ...common, + prompt_length: String(args.promptText.length), + prompt: args.promptText, + 'message.uuid': args.userUuid, + }), + logRecord('api_request_body', '2026-08-17T19:30:26.000Z', { + ...common, + body_ref: args.requestBodyPath, + request_id: args.requestId, + }), + logRecord('tool_decision', '2026-08-17T19:30:26.500Z', { + ...common, + tool_name: 'Read', + decision: 'reject', + source: 'user_reject', + }), + logRecord('tool_result', '2026-08-17T19:30:27.679Z', { + ...common, + tool_name: 'Read', + tool_use_id: 'toolu_smoke', + success: 'true', + duration_ms: '1', + }), + logRecord('hook_execution_start', '2026-08-17T19:30:28.000Z', { + ...common, + hook_name: 'hypaware-session-context', + hook_event: 'SessionStart', + }), + logRecord('hook_execution_complete', '2026-08-17T19:30:28.200Z', { + ...common, + hook_name: 'hypaware-session-context', + hook_event: 'SessionStart', + success: 'true', + duration_ms: '12', + }), + logRecord('api_request', '2026-08-17T19:30:31.009Z', { + ...common, + model: 'claude-haiku-4-5-20251001', + input_tokens: 73, + output_tokens: 113, + cache_read_tokens: 35212, + cache_creation_tokens: 307, + cost_usd: 0.0047732, + duration_ms: 1842, + request_id: args.requestId, + speed: 'normal', + query_source: 'sdk', + }), + logRecord('api_response_body', '2026-08-17T19:30:31.000Z', { + ...common, + body_ref: args.responseBodyPath, + request_id: args.requestId, + }), + logRecord('assistant_response', '2026-08-17T19:30:31.009Z', { + ...common, + response_length: args.responseText.length, + response: args.responseText, + request_id: args.requestId, + 'message.uuid': args.assistantUuid, + model: 'claude-haiku-4-5-20251001', + query_source: 'sdk', + }), + ], + }, + ], + }, + ], + } +} + +/** + * The evicted session's batch: the same wire shape, but its + * `api_request_body` names a file the startup sweep already removed, so + * only the event-carried content can land. + * + * @param {{ + * sessionId: string, + * userUuid: string, + * assistantUuid: string, + * requestId: string, + * promptText: string, + * responseText: string, + * requestBodyPath: string, + * }} args + */ +function buildEvictedSessionBatch(args) { + const common = { + 'session.id': args.sessionId, + 'app.version': '2.1.233', + 'app.entrypoint': 'sdk-cli', + 'user.account_uuid': 'c9f39145-595f-4b31-9c66-c5c658a80aed', + 'prompt.id': `p-${args.sessionId}`, + } + return { + resourceLogs: [ + { + resource: { attributes: kv({ 'service.name': 'claude-code', 'service.version': '2.1.233' }) }, + scopeLogs: [ + { + scope: { name: 'com.anthropic.claude_code.events', version: '2.1.233' }, + logRecords: [ + logRecord('user_prompt', '2026-08-17T19:40:01.000Z', { + ...common, + prompt_length: String(args.promptText.length), + prompt: args.promptText, + 'message.uuid': args.userUuid, + }), + logRecord('api_request_body', '2026-08-17T19:40:01.500Z', { + ...common, + body_ref: args.requestBodyPath, + request_id: args.requestId, + }), + logRecord('api_request', '2026-08-17T19:40:04.000Z', { + ...common, + model: 'claude-haiku-4-5-20251001', + input_tokens: 12, + output_tokens: 7, + request_id: args.requestId, + query_source: 'sdk', + }), + logRecord('assistant_response', '2026-08-17T19:40:04.100Z', { + ...common, + response_length: args.responseText.length, + response: args.responseText, + request_id: args.requestId, + 'message.uuid': args.assistantUuid, + model: 'claude-haiku-4-5-20251001', + query_source: 'sdk', + }), + ], + }, + ], + }, + ], + } +} + +/** + * The metrics half as Claude Code exports it: monotonic sums under the + * `com.anthropic.claude_code` meter scope, `session.id` on every data + * point, int64 values as strings on the OTLP/JSON wire. + * + * @param {{ sessionId: string }} args + */ +function buildMetricsBatch(args) { + const nanos = String(BigInt(Date.parse('2026-08-17T19:31:00.000Z')) * 1_000_000n) + return { + resourceMetrics: [ + { + resource: { attributes: kv({ 'service.name': 'claude-code', 'service.version': '2.1.233' }) }, + scopeMetrics: [ + { + scope: { name: 'com.anthropic.claude_code', version: '2.1.233' }, + metrics: [ + { + name: 'claude_code.cost.usage', + unit: 'USD', + sum: { + aggregationTemporality: 2, + isMonotonic: true, + dataPoints: [ + { + attributes: kv({ 'session.id': args.sessionId, model: 'claude-haiku-4-5-20251001' }), + timeUnixNano: nanos, + asDouble: 0.0047732, + }, + ], + }, + }, + { + name: 'claude_code.lines_of_code.count', + sum: { + aggregationTemporality: 2, + isMonotonic: true, + dataPoints: [ + { + attributes: kv({ 'session.id': args.sessionId, type: 'added' }), + timeUnixNano: nanos, + asInt: '42', + }, + ], + }, + }, + ], + }, + ], + }, + ], + } +} + +/** @param {string} file */ +async function fileExists(file) { + try { + await fs.stat(file) + return true + } catch { + return false + } +} + +/** + * @param {string} name + * @param {string} timestamp + * @param {Record} attrs + */ +function logRecord(name, timestamp, attrs) { + const nanos = String(BigInt(Date.parse(timestamp)) * 1_000_000n) + return { + timeUnixNano: nanos, + observedTimeUnixNano: nanos, + body: { stringValue: `claude_code.${name}` }, + attributes: kv({ ...attrs, 'event.name': name, 'event.timestamp': timestamp }), + } +} + +/** @param {Record} attrs */ +function kv(attrs) { + return Object.entries(attrs).map(([key, value]) => { + if (typeof value === 'number') { + return Number.isInteger(value) + ? { key, value: { intValue: value } } + : { key, value: { doubleValue: value } } + } + if (typeof value === 'boolean') return { key, value: { boolValue: value } } + return { key, value: { stringValue: String(value) } } + }) +} + +/** + * @param {string} url + * @param {unknown} payload + */ +async function postJson(url, payload) { + const response = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(payload), + }) + // Drain so the socket is released before the listener closes. + await response.text() + return response +} + +/** + * @param {{ sql: string, kernel: any, registry: any, env: any, expect: any, label: string }} args + * @returns {Promise} + */ +async function queryRows(args) { + const { sql, kernel, registry, env, expect, label } = args + const out = makeBuf() + const err = makeBuf() + // `--max-cell 0`: the display-value clip would truncate the long tool + // args this smoke exists to prove untruncated. + const code = await dispatch( + ['query', 'sql', sql, '--refresh', 'always', '--format', 'json', '--max-cell', '0', '--max-bytes', '0'], + { stdout: out, stderr: err, kernel, registry, env } + ) + expect.that(`dispatch: query (${label}) exited 0`, code, (v) => v === 0) + expect.that(`stderr: query (${label}) had no errors`, err.text(), (v) => typeof v === 'string' && v.length === 0) + try { + return JSON.parse(out.text()) + } catch (e) { + expect.that( + `stdout: query (${label}) was valid JSON (${e instanceof Error ? e.message : String(e)})`, + false, + (v) => v === true, + ) + return [] + } +} + +function makeBuf() { + /** @type {string[]} */ + const chunks = [] + return { + /** @param {unknown} chunk */ + write(chunk) { + chunks.push(typeof chunk === 'string' ? chunk : String(chunk)) + return true + }, + text() { + return chunks.join('') + }, + } +} diff --git a/hypaware-core/smoke/flows/claude_telemetry_hypignore_drop.js b/hypaware-core/smoke/flows/claude_telemetry_hypignore_drop.js new file mode 100644 index 00000000..6cf9eae4 --- /dev/null +++ b/hypaware-core/smoke/flows/claude_telemetry_hypignore_drop.js @@ -0,0 +1,621 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' +import process from 'node:process' +import { Readable } from 'node:stream' + +import { + Attr, + installObservability, + getLogger, + runRoot, +} from '../../../src/core/observability/index.js' +import { defaultConfigPath } from '../../../src/core/config/schema.js' +import { runDaemon } from '../../../src/core/daemon/runtime.js' +import { dispatch } from '../../../src/core/cli/dispatch.js' +import { localOnlyListPath, writeLocalOnlyEntries } from '../../../src/core/usage-policy/index.js' +import { claudeBodySpoolDir } from '../../plugins-workspace/claude/src/telemetry/spool.js' + +/** + * Hermetic smoke: the OTEL-path analog of `hypignore_capture_drop`. The folder + * usage policy decides at INGEST on the telemetry listener, before a row + * exists, and a dropped session's spooled bodies are deleted rather than left + * to age out of the cap. + * + * Boots the real daemon with `@hypaware/ai-gateway` + `@hypaware/claude` + * (telemetry listener on a dynamic port) and drives four sessions, each with a + * body staged in the spool the way Claude Code drops it: + * + * 1. `clean` - a cwd nothing governs. Its rows land and its body is + * projected then deleted, which is what makes the three negatives below + * mean something. + * 2. `ignored` - a cwd under a `.hypignore` holding `ignore`. Zero rows in + * either dataset, body gone, drop signal naming the governing file. + * 3. `private` - a cwd on the MACHINE-LOCAL list (LLP 0103) with no dotfile + * anywhere near it. Same outcome, governed by the list file, which is + * only reachable if the listener reads the list from the SHARED state + * root rather than its own per-plugin one. + * 4. `hookless` - no SessionStart record at all, so no cwd and no verdict. + * Withheld, not recorded: this is the fail-open window LLP 0085 patches, + * proven closed on this path rather than reopened. + * + * @param {{ harness: any, expect: any }} args + * @ref LLP 0257#testing [tests]: S25 - the privacy smoke for `.hypignore`: only + * clean rows land, the drop signal fires, and the ignored session's bodies + * are gone from the spool + * @ref LLP 0254#policy-inline [tests]: the check runs at ingest with cwd in + * hand, so no row is written before the policy resolves + * @ref LLP 0253#delete-on-drop [tests]: a dropped session's bodies are deleted, + * never merely skipped + */ +export async function run({ harness, expect }) { + const obs = installObservability() + if (!obs.tracer.provider) { + throw new Error( + 'claude_telemetry_hypignore_drop: tracer provider not installed - expected HYP_DEV_TELEMETRY=1' + ) + } + const log = getLogger('smoke') + + /** + * @param {string} name + * @returns {Record} + */ + const stepBag = (name) => ({ + [Attr.COMPONENT]: 'smoke', + [Attr.OPERATION]: 'step', + [Attr.SMOKE_NAME]: harness.smokeName, + [Attr.SMOKE_STEP]: name, + [Attr.DEV_RUN_ID]: harness.devRunId, + status: 'ok', + }) + + /** + * @template T + * @param {string} name + * @param {() => Promise} fn + * @returns {Promise} + */ + const step = (name, fn) => + runRoot(`smoke.step.${name}`, stepBag(name), async () => { + log.info(`smoke step ${name}`, stepBag(name)) + return fn() + }) + + const envSnapshot = { + HYP_HOME: process.env.HYP_HOME, + HYP_CONFIG: process.env.HYP_CONFIG, + HOME: process.env.HOME, + } + /** @type {Awaited> | undefined} */ + let handle + let obsShutDown = false + + const cleanSession = `clean-${harness.devRunId}` + const ignoredSession = `ignored-${harness.devRunId}` + const privateSession = `private-${harness.devRunId}` + const hooklessSession = `hookless-${harness.devRunId}` + + try { + // ----- smoke_step: setup ----- + const setup = await step('setup', async () => { + const claudeHome = path.join(harness.hypHome, 'home') + await fs.mkdir(path.join(claudeHome, '.claude', 'projects'), { recursive: true }) + + // Three scopes: one governed by a committed dotfile, one governed by the + // machine-local list only, one governed by nothing. + const ignoredCwd = path.join(harness.tmpDir, 'ignored-repo') + const privateCwd = path.join(harness.tmpDir, 'private-repo') + const cleanCwd = path.join(harness.tmpDir, 'clean-repo') + await fs.mkdir(ignoredCwd, { recursive: true }) + await fs.mkdir(privateCwd, { recursive: true }) + await fs.mkdir(cleanCwd, { recursive: true }) + const governingFile = path.join(ignoredCwd, '.hypignore') + await fs.writeFile(governingFile, '# self-documenting\nignore\n', 'utf8') + + // The machine-local half, written where `hyp ignore --private` writes it: + // the SHARED state root, not the plugin's own state directory. + const stateRoot = path.join(harness.hypHome, 'hypaware') + await writeLocalOnlyEntries({ + stateDir: stateRoot, + entries: [{ dir: privateCwd, class: 'ignore' }], + }) + + const configPath = defaultConfigPath(harness.hypHome) + await fs.mkdir(path.dirname(configPath), { recursive: true }) + await fs.writeFile(configPath, JSON.stringify({ + version: 2, + plugins: [ + { name: '@hypaware/ai-gateway', config: { listen: '127.0.0.1:0' } }, + { + name: '@hypaware/claude', + config: { telemetry: { listen_host: '127.0.0.1', listen_port: 0 } }, + }, + ], + query: { cache: { retention: { default_days: 30 } } }, + }, null, 2)) + + process.env.HYP_HOME = harness.hypHome + process.env.HYP_CONFIG = configPath + process.env.HOME = claudeHome + + return { configPath, stateRoot, ignoredCwd, privateCwd, cleanCwd, governingFile } + }) + const { configPath, stateRoot, ignoredCwd, privateCwd, cleanCwd, governingFile } = setup + const env = { ...process.env, HYP_HOME: harness.hypHome, HYP_CONFIG: configPath } + const spoolDir = claudeBodySpoolDir(harness.hypHome) + const sessionContextFile = path.join( + stateRoot, 'plugins', '@hypaware/claude', 'session-context.jsonl' + ) + + // ----- smoke_step: boot_and_stage ----- + const listenerUrl = await step('boot_and_stage', async () => { + handle = await runDaemon({ + hypHome: harness.hypHome, + configPath, + env: process.env, + runId: harness.devRunId, + tickIntervalMs: 50, + installSignalHandlers: false, + }) + const snapshot = handle.snapshot() + const details = /** @type {{ listen_host: string, listen_port: number }} */ ( + snapshot.sources.find((s) => s.name === 'claude-telemetry')?.details + ) + expect.that( + 'snapshot: the claude listener reports a bound port', + details, + (v) => v !== undefined && typeof v.listen_port === 'number' && v.listen_port > 0, + ) + + // The SessionStart hook records, exactly as an attached Claude Code would + // have written them. `hookless` deliberately gets none. + for (const [sessionId, cwd] of [ + [cleanSession, cleanCwd], + [ignoredSession, ignoredCwd], + [privateSession, privateCwd], + ]) { + const hookCode = await dispatch( + ['claude-hook', 'session-context', '--state-file', sessionContextFile], + { + stdout: makeBuf(), + stderr: makeBuf(), + stdin: /** @type {any} */ (streamOf(JSON.stringify({ + session_id: sessionId, + cwd, + hook_event_name: 'SessionStart', + }))), + env, + } + ) + expect.that(`hook: session-context for ${sessionId} exited 0`, hookCode, (v) => v === 0) + } + + return `http://${details.listen_host}:${details.listen_port}` + }) + + // ----- smoke_step: post_batches ----- + await step('post_batches', async () => { + /** @type {Record} */ + const bodies = {} + for (const sessionId of [cleanSession, ignoredSession, privateSession, hooklessSession]) { + const bodyPath = path.join(spoolDir, `${sessionId}-req.json`) + await fs.mkdir(spoolDir, { recursive: true }) + await fs.writeFile(bodyPath, JSON.stringify({ + model: 'claude-haiku-4-5-20251001', + system: [{ type: 'text', text: `system prompt for ${sessionId}` }], + messages: [{ role: 'user', content: `body text for ${sessionId}` }], + }), 'utf8') + bodies[sessionId] = bodyPath + } + + for (const sessionId of [cleanSession, ignoredSession, privateSession, hooklessSession]) { + const posted = await postJson(`${listenerUrl}/v1/logs`, turnBatch({ + sessionId, + userUuid: `u-${sessionId}-user`, + assistantUuid: `u-${sessionId}-asst`, + requestId: `req-${sessionId}`, + promptText: `prompt from ${sessionId}`, + responseText: `reply to ${sessionId}`, + bodyPath: bodies[sessionId], + withToolDecision: true, + })) + expect.that(`listener: the POST for ${sessionId} returned 200`, posted.status, (v) => v === 200) + } + + // A cost data point for the ignored session: the behavioral half of the + // record is governed by the same verdict. + const postedMetrics = await postJson(`${listenerUrl}/v1/metrics`, metricsBatch({ + sessionId: ignoredSession, + })) + expect.that('listener: the ignored session\'s metrics POST returned 200', postedMetrics.status, (v) => v === 200) + + // The three suppressed sessions' bodies are GONE; the clean one's was + // projected and then deleted, which is the same file state reached two + // different ways - so the cache assertions below are what tell them apart. + for (const sessionId of [ignoredSession, privateSession, hooklessSession]) { + expect.that( + `spool: ${sessionId}'s body was deleted unread`, + await fileExists(bodies[sessionId]), + (v) => v === false, + ) + } + expect.that( + 'spool: the clean session\'s body was projected then deleted', + await fileExists(bodies[cleanSession]), + (v) => v === false, + ) + expect.that( + 'spool: nothing is left in the spool directory', + await fs.readdir(spoolDir), + (v) => Array.isArray(v) && v.length === 0, + ) + }) + + // ----- Shut down + flush so the cache and JSONL artifacts are complete ----- + await handle?.stop() + await handle?.done + handle = undefined + await obs.shutdown() + obsShutDown = true + + // ----- smoke_step: assert_cache ----- + await step('assert_cache', async () => { + const messageRows = await queryRows({ + sql: ` + select session_id, role, content_text, system_text, cwd + from ai_gateway_messages + order by session_id, message_index + `.trim().replace(/\s+/g, ' '), + env, + expect, + label: 'message rows', + }) + expect.that( + 'query: only the clean session recorded', + [...new Set(messageRows.map((/** @type {any} */ r) => r.session_id))], + (v) => JSON.stringify(v) === JSON.stringify([cleanSession]), + ) + expect.that( + 'query: the clean rows carry the cwd the hook recorded', + messageRows, + (v) => v.length > 0 && v.every((/** @type {any} */ r) => r.cwd === cleanCwd), + ) + expect.that( + 'query: no row carries a suppressed session\'s content or system text', + messageRows, + (v) => v.every((/** @type {any} */ r) => + !(r.content_text ?? '').includes(ignoredSession) && + !(r.content_text ?? '').includes(privateSession) && + !(r.content_text ?? '').includes(hooklessSession) && + !(r.system_text ?? '').includes(ignoredSession) && + !(r.system_text ?? '').includes(privateSession) && + !(r.system_text ?? '').includes(hooklessSession)), + ) + + const eventRows = await queryRows({ + sql: 'select session_id, event_name from claude_telemetry_events order by session_id, event_name', + env, + expect, + label: 'behavioral rows', + }) + expect.that( + 'events: only the clean session\'s behavioral rows landed', + [...new Set(eventRows.map((/** @type {any} */ r) => r.session_id))], + (v) => JSON.stringify(v) === JSON.stringify([cleanSession]), + ) + }) + + // ----- smoke_step: assert_signals ----- + await step('assert_signals', async () => { + const logs = await expect.logs() + const drops = logs.filter( + (/** @type {any} */ l) => l.body === 'claude.telemetry.usage_policy_drop', + ) + + // @ref LLP 0257#observability [tests]: the policy drop emits a structured + // signal naming what governed it. + const dotfileDrops = drops.filter( + (/** @type {any} */ l) => l.attributes?.session_id === ignoredSession, + ) + expect.that( + 'logs: the .hypignore drop fired for the events batch and the metrics batch, naming the governing file', + dotfileDrops, + (v) => v.length === 2 && v.every((/** @type {any} */ l) => + l.attributes?.policy_source === 'usage_policy' && + l.attributes?.governed_by === governingFile), + ) + expect.that( + 'logs: the .hypignore drop deleted the session\'s body', + dotfileDrops.find((/** @type {any} */ l) => Number(l.attributes?.bodies_deleted) === 1), + (v) => v !== undefined, + ) + + const listDrop = drops.find( + (/** @type {any} */ l) => l.attributes?.session_id === privateSession, + ) + expect.that( + 'logs: the machine-local list drop names the list at the SHARED state root', + listDrop, + (v) => v !== undefined && v.attributes?.policy_source === 'usage_policy' && + v.attributes?.governed_by === localOnlyListPath(stateRoot) && + Number(v.attributes?.bodies_deleted) === 1, + ) + + const withheld = drops.find( + (/** @type {any} */ l) => l.attributes?.session_id === hooklessSession, + ) + expect.that( + 'logs: the session with no hook record was withheld as undetermined, not recorded', + withheld, + (v) => v !== undefined && v.attributes?.policy_source === 'undetermined_cwd' && + v.attributes?.recovery === 'transcript_backfill', + ) + + const traces = await expect.traces() + const receives = traces.filter((/** @type {any} */ t) => t.name === 'claude.telemetry.receive') + const suppressed = receives.filter( + (/** @type {any} */ t) => + Number(t.attributes?.events_dropped ?? 0) > 0 || + Number(t.attributes?.events_undetermined ?? 0) > 0, + ) + expect.that( + 'traces: three suppressed batches plus the ignored session\'s metrics batch', + suppressed, + (v) => v.length === 4, + ) + expect.that( + 'traces: every suppressed batch wrote zero message rows', + suppressed, + (v) => v.every((/** @type {any} */ t) => Number(t.attributes?.row_count) === 0), + ) + expect.that( + 'traces: the clean batch wrote rows and dropped nothing', + receives.filter((/** @type {any} */ t) => Number(t.attributes?.row_count) > 0), + (v) => v.length === 1 && v[0].attributes?.events_dropped === undefined && + v[0].attributes?.events_undetermined === undefined, + ) + }) + } finally { + if (handle) { + try { await handle.stop() } catch { /* already stopping or stopped */ } + try { await handle.done } catch { /* surface the original failure */ } + } + if (!obsShutDown) { + try { await obs.shutdown() } catch { /* best-effort flush */ } + } + restoreEnv('HYP_HOME', envSnapshot.HYP_HOME) + restoreEnv('HYP_CONFIG', envSnapshot.HYP_CONFIG) + restoreEnv('HOME', envSnapshot.HOME) + } +} + +/** + * @param {string} key + * @param {string | undefined} value + */ +function restoreEnv(key, value) { + if (value === undefined) delete process.env[key] + else process.env[key] = value +} + +// --------------------------------------------------------------------- +// Fixtures + helpers (mirrors claude_telemetry_session_ignore) +// --------------------------------------------------------------------- + +/** + * One turn as Claude Code exports it: content events, the `api_request` that + * carries usage, an `api_request_body` pointing into the spool, and a + * behavioral event. + * + * @param {{ + * sessionId: string, + * userUuid: string, + * assistantUuid: string, + * requestId: string, + * promptText: string, + * responseText: string, + * bodyPath?: string, + * withToolDecision?: boolean, + * }} args + */ +function turnBatch(args) { + const common = { + 'session.id': args.sessionId, + 'app.version': '2.1.233', + 'app.entrypoint': 'sdk-cli', + 'user.account_uuid': 'c9f39145-595f-4b31-9c66-c5c658a80aed', + 'terminal.type': 'ghostty', + 'prompt.id': `p-${args.sessionId}`, + } + const records = [ + logRecord('user_prompt', '2026-08-17T20:30:24.450Z', { + ...common, + prompt_length: String(args.promptText.length), + prompt: args.promptText, + 'message.uuid': args.userUuid, + }), + ...(args.bodyPath + ? [logRecord('api_request_body', '2026-08-17T20:30:26.000Z', { + ...common, + body_ref: args.bodyPath, + request_id: args.requestId, + })] + : []), + ...(args.withToolDecision + ? [logRecord('tool_decision', '2026-08-17T20:30:26.500Z', { + ...common, + tool_name: 'Read', + decision: 'accept', + source: 'config', + })] + : []), + logRecord('api_request', '2026-08-17T20:30:31.009Z', { + ...common, + model: 'claude-haiku-4-5-20251001', + input_tokens: 73, + output_tokens: 113, + request_id: args.requestId, + query_source: 'sdk', + }), + logRecord('assistant_response', '2026-08-17T20:30:31.100Z', { + ...common, + response_length: args.responseText.length, + response: args.responseText, + request_id: args.requestId, + 'message.uuid': args.assistantUuid, + model: 'claude-haiku-4-5-20251001', + query_source: 'sdk', + }), + ] + return { + resourceLogs: [ + { + resource: { attributes: kv({ 'service.name': 'claude-code', 'service.version': '2.1.233' }) }, + scopeLogs: [ + { + scope: { name: 'com.anthropic.claude_code.events', version: '2.1.233' }, + logRecords: records, + }, + ], + }, + ], + } +} + +/** + * One cost data point under the Claude Code meter scope. + * + * @param {{ sessionId: string }} args + */ +function metricsBatch(args) { + const nanos = String(BigInt(Date.parse('2026-08-17T20:31:00.000Z')) * 1_000_000n) + return { + resourceMetrics: [ + { + resource: { attributes: kv({ 'service.name': 'claude-code', 'service.version': '2.1.233' }) }, + scopeMetrics: [ + { + scope: { name: 'com.anthropic.claude_code', version: '2.1.233' }, + metrics: [ + { + name: 'claude_code.cost.usage', + unit: 'USD', + sum: { + aggregationTemporality: 2, + isMonotonic: true, + dataPoints: [ + { + attributes: kv({ 'session.id': args.sessionId, model: 'claude-haiku-4-5-20251001' }), + timeUnixNano: nanos, + asDouble: 0.0047732, + }, + ], + }, + }, + ], + }, + ], + }, + ], + } +} + +/** + * @param {string} name + * @param {string} timestamp + * @param {Record} attrs + */ +function logRecord(name, timestamp, attrs) { + const nanos = String(BigInt(Date.parse(timestamp)) * 1_000_000n) + return { + timeUnixNano: nanos, + observedTimeUnixNano: nanos, + body: { stringValue: `claude_code.${name}` }, + attributes: kv({ ...attrs, 'event.name': name, 'event.timestamp': timestamp }), + } +} + +/** @param {Record} attrs */ +function kv(attrs) { + return Object.entries(attrs).map(([key, value]) => { + if (typeof value === 'number') { + return Number.isInteger(value) + ? { key, value: { intValue: value } } + : { key, value: { doubleValue: value } } + } + if (typeof value === 'boolean') return { key, value: { boolValue: value } } + return { key, value: { stringValue: String(value) } } + }) +} + +/** + * @param {string} url + * @param {unknown} payload + */ +async function postJson(url, payload) { + const response = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(payload), + }) + // Drain so the socket is released before the listener closes. + await response.text() + return response +} + +/** @param {string} file */ +async function fileExists(file) { + try { + await fs.stat(file) + return true + } catch { + return false + } +} + +/** @param {string} text */ +function streamOf(text) { + return Readable.from([text]) +} + +/** + * @param {{ sql: string, env: any, expect: any, label: string }} args + * @returns {Promise} + */ +async function queryRows({ sql, env, expect, label }) { + const out = makeBuf() + const err = makeBuf() + const code = await dispatch( + ['query', 'sql', sql, '--refresh', 'always', '--format', 'json', '--max-bytes', '0'], + { stdout: out, stderr: err, env } + ) + expect.that(`dispatch: query (${label}) exited 0`, code, (v) => v === 0) + expect.that(`stderr: query (${label}) had no errors`, err.text(), (v) => typeof v === 'string' && v.length === 0) + try { + return JSON.parse(out.text()) + } catch (e) { + expect.that( + `stdout: query (${label}) was valid JSON (${e instanceof Error ? e.message : String(e)})`, + false, + (v) => v === true, + ) + return [] + } +} + +function makeBuf() { + /** @type {string[]} */ + const chunks = [] + return { + /** @param {unknown} chunk */ + write(chunk) { + chunks.push(typeof chunk === 'string' ? chunk : String(chunk)) + return true + }, + text() { + return chunks.join('') + }, + } +} diff --git a/hypaware-core/smoke/flows/claude_telemetry_session_ignore.js b/hypaware-core/smoke/flows/claude_telemetry_session_ignore.js new file mode 100644 index 00000000..b318589a --- /dev/null +++ b/hypaware-core/smoke/flows/claude_telemetry_session_ignore.js @@ -0,0 +1,697 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' +import process from 'node:process' +import { Readable } from 'node:stream' + +import { + Attr, + installObservability, + getLogger, + runRoot, +} from '../../../src/core/observability/index.js' +import { defaultConfigPath } from '../../../src/core/config/schema.js' +import { runDaemon } from '../../../src/core/daemon/runtime.js' +import { dispatch } from '../../../src/core/cli/dispatch.js' +import { claudeBodySpoolDir } from '../../plugins-workspace/claude/src/telemetry/spool.js' + +/** + * Hermetic smoke: the per-session opt-out (LLP 0066) reaching the claude + * telemetry listener (LLP 0256) end to end, beside the gateway. + * + * Boots the real daemon with `@hypaware/ai-gateway` + `@hypaware/claude` + * (telemetry listener on a dynamic port), then: + * + * 1. `hyp session ignore --json` - the receipt must report BOTH + * recorders (the gateway by its own resolution, the listener by its + * `control_routes` advertisement in the live snapshot), and a direct + * `GET` on each control route must confirm membership, proving the + * gateway's own route is unaffected by the second host. + * 2. POST the ignored session's OTLP/JSON events (content, behavioral, + * and body-pointer events naming a staged spool file) plus a metrics + * batch. NOTHING may land: zero `ai_gateway_messages` rows, zero + * `claude_telemetry_events` rows, and the spooled body DELETED unread + * (LLP 0253 #delete-on-drop), with the `usage_policy_drop` signal + * naming `session_opt_out`. + * 3. A clean session's batch lands normally, isolating the drop. + * 4. `hyp session unignore --json` (both recorders again), then a + * resumed batch for the SAME session: its rows land and its body is + * projected-then-deleted, proving capture restores. + * + * The gateway's own capture-seam drop under this route stays pinned by + * `session_optout_capture_drop`, which runs unchanged. + * + * @param {{ harness: any, expect: any }} args + * @ref LLP 0257#testing [tests]: S25 - the privacy smoke for the control + * route: only clean rows land, the drop signal fires, and the ignored + * session's bodies are gone from the spool + * @ref LLP 0256#cli-posts-to-both [tests]: ignoring via the CLI reaches both + * servers and reports each outcome + */ +export async function run({ harness, expect }) { + const obs = installObservability() + if (!obs.tracer.provider) { + throw new Error( + 'claude_telemetry_session_ignore: tracer provider not installed - expected HYP_DEV_TELEMETRY=1' + ) + } + const log = getLogger('smoke') + + /** + * @param {string} name + * @returns {Record} + */ + const stepBag = (name) => ({ + [Attr.COMPONENT]: 'smoke', + [Attr.OPERATION]: 'step', + [Attr.SMOKE_NAME]: harness.smokeName, + [Attr.SMOKE_STEP]: name, + [Attr.DEV_RUN_ID]: harness.devRunId, + status: 'ok', + }) + + /** + * @template T + * @param {string} name + * @param {() => Promise} fn + * @returns {Promise} + */ + const step = (name, fn) => + runRoot(`smoke.step.${name}`, stepBag(name), async () => { + log.info(`smoke step ${name}`, stepBag(name)) + return fn() + }) + + const envSnapshot = { + HYP_HOME: process.env.HYP_HOME, + HYP_CONFIG: process.env.HYP_CONFIG, + HOME: process.env.HOME, + } + /** @type {Awaited> | undefined} */ + let handle + let obsShutDown = false + + const ignoredSession = `optout-otel-${harness.devRunId}` + const cleanSession = `clean-otel-${harness.devRunId}` + + try { + // ----- smoke_step: setup ----- + const setup = await step('setup', async () => { + const claudeHome = path.join(harness.hypHome, 'home') + await fs.mkdir(path.join(claudeHome, '.claude', 'projects'), { recursive: true }) + + const configPath = defaultConfigPath(harness.hypHome) + await fs.mkdir(path.dirname(configPath), { recursive: true }) + await fs.writeFile(configPath, JSON.stringify({ + version: 2, + plugins: [ + { name: '@hypaware/ai-gateway', config: { listen: '127.0.0.1:0' } }, + { + name: '@hypaware/claude', + config: { telemetry: { listen_host: '127.0.0.1', listen_port: 0 } }, + }, + ], + query: { cache: { retention: { default_days: 30 } } }, + }, null, 2)) + + process.env.HYP_HOME = harness.hypHome + process.env.HYP_CONFIG = configPath + process.env.HOME = claudeHome + + return { configPath } + }) + const { configPath } = setup + const env = { ...process.env, HYP_HOME: harness.hypHome, HYP_CONFIG: configPath } + const spoolDir = claudeBodySpoolDir(harness.hypHome) + const stateRoot = path.join(harness.hypHome, 'hypaware') + const sessionContextFile = path.join( + stateRoot, 'plugins', '@hypaware/claude', 'session-context.jsonl' + ) + + // ----- smoke_step: boot_and_ignore ----- + const endpoints = await step('boot_and_ignore', async () => { + handle = await runDaemon({ + hypHome: harness.hypHome, + configPath, + env: process.env, + runId: harness.devRunId, + tickIntervalMs: 50, + installSignalHandlers: false, + }) + const snapshot = handle.snapshot() + const gatewayDetails = /** @type {{ host: string, port: number }} */ ( + snapshot.sources.find((s) => s.name === 'ai-gateway')?.details + ) + const listenerDetails = /** @type {{ listen_host: string, listen_port: number, control_routes?: string[] }} */ ( + snapshot.sources.find((s) => s.name === 'claude-telemetry')?.details + ) + expect.that( + 'snapshot: the gateway reports a bound port', + gatewayDetails, + (v) => v !== undefined && typeof v.port === 'number' && v.port > 0, + ) + expect.that( + 'snapshot: the claude listener reports a bound port and advertises the ignore route', + listenerDetails, + (v) => v !== undefined && typeof v.listen_port === 'number' && v.listen_port > 0 && + Array.isArray(v.control_routes) && v.control_routes.includes('ignore/session'), + ) + const gatewayUrl = `http://${gatewayDetails.host}:${gatewayDetails.port}` + const listenerUrl = `http://${listenerDetails.listen_host}:${listenerDetails.listen_port}` + + // SessionStart hook records for both sessions, exactly as an attached + // Claude Code would have written them (the source of cwd identity). + for (const sessionId of [ignoredSession, cleanSession]) { + const hookOut = makeBuf() + const hookCode = await dispatch( + ['claude-hook', 'session-context', '--state-file', sessionContextFile], + { + stdout: hookOut, + stderr: makeBuf(), + stdin: /** @type {any} */ (streamOf(JSON.stringify({ + session_id: sessionId, + cwd: harness.tmpDir, + hook_event_name: 'SessionStart', + }))), + env, + } + ) + expect.that(`hook: session-context for ${sessionId} exited 0`, hookCode, (v) => v === 0) + } + + // The CLI mutation must reach BOTH recorders and say so. + const ignoreOut = makeBuf() + const ignoreErr = makeBuf() + const ignoreCode = await dispatch( + ['session', 'ignore', ignoredSession, '--json'], + { stdout: ignoreOut, stderr: ignoreErr, env } + ) + expect.that('cli: hyp session ignore exited 0', ignoreCode, (v) => v === 0) + const receipt = JSON.parse(ignoreOut.text()) + expect.that( + 'cli: the receipt is ok with set_membership as its guarantee', + receipt, + (v) => v.status === 'ok' && v.guarantee === 'set_membership' && v.ignored === true, + ) + expect.that( + 'cli: the receipt reports BOTH recorders, gateway first', + receipt.recorders, + (v) => Array.isArray(v) && v.length === 2 && + v[0].recorder === 'gateway' && v[0].status === 'ok' && v[0].endpoint === gatewayUrl && + v[1].recorder === 'claude-telemetry' && v[1].status === 'ok' && v[1].endpoint === listenerUrl, + ) + + // Membership confirmed on each route directly: the second host did not + // disturb the gateway's own route, and the listener really holds the id. + const onGateway = await controlGet(gatewayUrl, ignoredSession) + expect.that( + 'control: the gateway route confirms membership', + onGateway, + (v) => v.status === 200 && v.body.ignored === true && v.body.total === 1, + ) + const onListener = await controlGet(listenerUrl, ignoredSession) + expect.that( + 'control: the listener route confirms membership', + onListener, + (v) => v.status === 200 && v.body.ignored === true && v.body.total === 1, + ) + + return { gatewayUrl, listenerUrl } + }) + const { listenerUrl } = endpoints + + // ----- smoke_step: dropped_batch ----- + await step('dropped_batch', async () => { + // The ignored session's body, staged the way Claude Code drops it. + const droppedBodyPath = path.join(spoolDir, `${ignoredSession}-req.json`) + await fs.mkdir(spoolDir, { recursive: true }) + await fs.writeFile(droppedBodyPath, JSON.stringify({ + model: 'claude-haiku-4-5-20251001', + system: [{ type: 'text', text: 'You are the ignored session.' }], + messages: [{ role: 'user', content: `dropped ${harness.devRunId}` }], + }), 'utf8') + + const posted = await postJson(`${listenerUrl}/v1/logs`, turnBatch({ + sessionId: ignoredSession, + userUuid: `u-drop-user-${harness.devRunId}`, + assistantUuid: `u-drop-asst-${harness.devRunId}`, + requestId: `req-drop-${harness.devRunId}`, + promptText: `dropped ${harness.devRunId}`, + responseText: 'dropped reply', + bodyPath: droppedBodyPath, + withToolDecision: true, + })) + expect.that('listener: the ignored session\'s POST returned 200', posted.status, (v) => v === 200) + expect.that( + 'spool: the ignored session\'s body was DELETED, not skipped', + await fileExists(droppedBodyPath), + (v) => v === false, + ) + + const postedMetrics = await postJson(`${listenerUrl}/v1/metrics`, metricsBatch({ + sessionId: ignoredSession, + })) + expect.that('listener: the ignored session\'s metrics POST returned 200', postedMetrics.status, (v) => v === 200) + + // The clean session lands normally beside the drop. + const cleanPosted = await postJson(`${listenerUrl}/v1/logs`, turnBatch({ + sessionId: cleanSession, + userUuid: `u-clean-user-${harness.devRunId}`, + assistantUuid: `u-clean-asst-${harness.devRunId}`, + requestId: `req-clean-${harness.devRunId}`, + promptText: `clean ${harness.devRunId}`, + responseText: 'clean reply', + })) + expect.that('listener: the clean session\'s POST returned 200', cleanPosted.status, (v) => v === 200) + }) + + // ----- smoke_step: unignore_and_resume ----- + await step('unignore_and_resume', async () => { + const unignoreOut = makeBuf() + const unignoreCode = await dispatch( + ['session', 'unignore', ignoredSession, '--json'], + { stdout: unignoreOut, stderr: makeBuf(), env } + ) + expect.that('cli: hyp session unignore exited 0', unignoreCode, (v) => v === 0) + const receipt = JSON.parse(unignoreOut.text()) + expect.that( + 'cli: the unignore receipt reports both recorders released the id', + receipt, + (v) => v.status === 'ok' && Array.isArray(v.recorders) && v.recorders.length === 2 && + v.recorders.every((/** @type {any} */ r) => r.status === 'ok' && r.ignored === false), + ) + const onListener = await controlGet(listenerUrl, ignoredSession) + expect.that( + 'control: the listener route confirms the removal', + onListener, + (v) => v.status === 200 && v.body.ignored === false && v.body.total === 0, + ) + + // The SAME session records again: capture restored, body projected + // then deleted like any other. + const resumedBodyPath = path.join(spoolDir, `${ignoredSession}-resumed-req.json`) + await fs.writeFile(resumedBodyPath, JSON.stringify({ + model: 'claude-haiku-4-5-20251001', + system: [{ type: 'text', text: 'You are the resumed session.' }], + messages: [{ role: 'user', content: `resumed ${harness.devRunId}` }], + }), 'utf8') + const resumed = await postJson(`${listenerUrl}/v1/logs`, turnBatch({ + sessionId: ignoredSession, + userUuid: `u-resume-user-${harness.devRunId}`, + assistantUuid: `u-resume-asst-${harness.devRunId}`, + requestId: `req-resume-${harness.devRunId}`, + promptText: `resumed ${harness.devRunId}`, + responseText: 'resumed reply', + bodyPath: resumedBodyPath, + })) + expect.that('listener: the resumed POST returned 200', resumed.status, (v) => v === 200) + expect.that( + 'spool: the resumed body was projected then deleted', + await fileExists(resumedBodyPath), + (v) => v === false, + ) + }) + + // ----- Shut down + flush so the cache and JSONL artifacts are complete ----- + await handle?.stop() + await handle?.done + handle = undefined + await obs.shutdown() + obsShutDown = true + + // ----- smoke_step: assert_cache ----- + await step('assert_cache', async () => { + const messageRows = await queryRows({ + sql: ` + select session_id, role, content_text, system_text + from ai_gateway_messages + where session_id in ('${ignoredSession}', '${cleanSession}') + order by session_id, message_index + `.trim().replace(/\s+/g, ' '), + env, + expect, + label: 'message rows', + }) + // 2 clean rows + 2 resumed rows; the dropped exchange contributes + // NOTHING, in content or in system text. + expect.that( + 'query: exactly four rows landed (clean turn + resumed turn)', + messageRows, + (v) => Array.isArray(v) && v.length === 4, + ) + expect.that( + 'query: no landed row carries the dropped content or its system text', + messageRows, + (v) => v.every((/** @type {any} */ r) => + !(r.content_text ?? '').includes('dropped') && + !(r.system_text ?? '').includes('ignored session')), + ) + const resumedRows = messageRows.filter((/** @type {any} */ r) => r.session_id === ignoredSession) + expect.that( + 'query: the formerly ignored session recorded the RESUMED turn only', + resumedRows, + (v) => v.length === 2 && v.some((r) => (r.content_text ?? '').includes('resumed')) && + v.every((r) => (r.system_text ?? '') === 'You are the resumed session.'), + ) + + const eventRows = await queryRows({ + sql: ` + select session_id, event_name + from claude_telemetry_events + where session_id in ('${ignoredSession}', '${cleanSession}') + order by session_id, event_name + `.trim().replace(/\s+/g, ' '), + env, + expect, + label: 'behavioral rows', + }) + // One api_request per landed turn. The dropped batch's tool_decision, + // api_request, and metric data point never became rows. + expect.that( + 'events: only the clean and resumed api_request rows landed', + eventRows.map((/** @type {any} */ r) => `${r.session_id}:${r.event_name}`), + (v) => JSON.stringify(v) === JSON.stringify([ + `${cleanSession}:api_request`, + `${ignoredSession}:api_request`, + ]), + ) + }) + + // ----- smoke_step: assert_signals ----- + await step('assert_signals', async () => { + const logs = await expect.logs() + + // @ref LLP 0257#observability [tests]: the policy drop and the + // control-route mutation both emit structured signals. + const drops = logs.filter( + (/** @type {any} */ l) => + l.body === 'claude.telemetry.usage_policy_drop' && + l.attributes?.session_id === ignoredSession, + ) + expect.that( + 'logs: the listener logged the opt-out drop for the events batch and the metrics batch', + drops, + (v) => Array.isArray(v) && v.length === 2 && + v.every((l) => l.attributes?.policy_source === 'session_opt_out'), + ) + const eventsDrop = drops.find((/** @type {any} */ l) => Number(l.attributes?.events_dropped) === 5) + expect.that( + 'logs: the events-batch drop counted five events and one deleted body', + eventsDrop, + (v) => v !== undefined && Number(v.attributes?.bodies_deleted) === 1, + ) + const metricsDrop = drops.find((/** @type {any} */ l) => Number(l.attributes?.events_dropped) === 1) + expect.that( + 'logs: the metrics-batch drop counted its one data point', + metricsDrop, + (v) => v !== undefined && Number(v.attributes?.bodies_deleted) === 0, + ) + + const mutations = logs.filter( + (/** @type {any} */ l) => l.body === 'claude.telemetry.control.ignore_session', + ) + expect.that( + 'logs: the listener logged both control mutations (POST then DELETE)', + mutations.map((/** @type {any} */ l) => l.attributes?.method), + (v) => JSON.stringify(v) === JSON.stringify(['POST', 'DELETE']), + ) + + const traces = await expect.traces() + const dropSpans = traces.filter( + (/** @type {any} */ t) => + t.name === 'claude.telemetry.receive' && Number(t.attributes?.events_dropped) > 0, + ) + expect.that( + 'traces: the receive spans carry the drop counts (events batch + metrics batch)', + dropSpans, + (v) => Array.isArray(v) && v.length === 2, + ) + const eventsSpan = dropSpans.find((/** @type {any} */ t) => t.attributes?.signal === 'logs') + expect.that( + 'traces: the dropped events batch wrote zero rows', + eventsSpan?.attributes, + (v) => v !== undefined && Number(v.events_dropped) === 5 && + Number(v.bodies_dropped) === 1 && Number(v.row_count) === 0, + ) + }) + } finally { + if (handle) { + try { await handle.stop() } catch { /* already stopping or stopped */ } + try { await handle.done } catch { /* surface the original failure */ } + } + if (!obsShutDown) { + try { await obs.shutdown() } catch { /* best-effort flush */ } + } + restoreEnv('HYP_HOME', envSnapshot.HYP_HOME) + restoreEnv('HYP_CONFIG', envSnapshot.HYP_CONFIG) + restoreEnv('HOME', envSnapshot.HOME) + } +} + +/** + * @param {string} key + * @param {string | undefined} value + */ +function restoreEnv(key, value) { + if (value === undefined) delete process.env[key] + else process.env[key] = value +} + +// --------------------------------------------------------------------- +// Fixtures + helpers (mirrors claude_telemetry_capture) +// --------------------------------------------------------------------- + +/** + * One turn as Claude Code exports it: content events, the `api_request` + * that carries usage, optionally an `api_request_body` pointing into the + * spool and a `tool_decision` behavioral event. + * + * @param {{ + * sessionId: string, + * userUuid: string, + * assistantUuid: string, + * requestId: string, + * promptText: string, + * responseText: string, + * bodyPath?: string, + * withToolDecision?: boolean, + * }} args + */ +function turnBatch(args) { + const common = { + 'session.id': args.sessionId, + 'app.version': '2.1.233', + 'app.entrypoint': 'sdk-cli', + 'user.account_uuid': 'c9f39145-595f-4b31-9c66-c5c658a80aed', + 'terminal.type': 'ghostty', + 'prompt.id': `p-${args.sessionId}`, + } + const records = [ + logRecord('user_prompt', '2026-08-17T20:30:24.450Z', { + ...common, + prompt_length: String(args.promptText.length), + prompt: args.promptText, + 'message.uuid': args.userUuid, + }), + ...(args.bodyPath + ? [logRecord('api_request_body', '2026-08-17T20:30:26.000Z', { + ...common, + body_ref: args.bodyPath, + request_id: args.requestId, + })] + : []), + ...(args.withToolDecision + ? [logRecord('tool_decision', '2026-08-17T20:30:26.500Z', { + ...common, + tool_name: 'Read', + decision: 'reject', + source: 'user_reject', + })] + : []), + logRecord('api_request', '2026-08-17T20:30:31.009Z', { + ...common, + model: 'claude-haiku-4-5-20251001', + input_tokens: 73, + output_tokens: 113, + request_id: args.requestId, + query_source: 'sdk', + }), + logRecord('assistant_response', '2026-08-17T20:30:31.100Z', { + ...common, + response_length: args.responseText.length, + response: args.responseText, + request_id: args.requestId, + 'message.uuid': args.assistantUuid, + model: 'claude-haiku-4-5-20251001', + query_source: 'sdk', + }), + ] + return { + resourceLogs: [ + { + resource: { attributes: kv({ 'service.name': 'claude-code', 'service.version': '2.1.233' }) }, + scopeLogs: [ + { + scope: { name: 'com.anthropic.claude_code.events', version: '2.1.233' }, + logRecords: records, + }, + ], + }, + ], + } +} + +/** + * One cost data point under the Claude Code meter scope. + * + * @param {{ sessionId: string }} args + */ +function metricsBatch(args) { + const nanos = String(BigInt(Date.parse('2026-08-17T20:31:00.000Z')) * 1_000_000n) + return { + resourceMetrics: [ + { + resource: { attributes: kv({ 'service.name': 'claude-code', 'service.version': '2.1.233' }) }, + scopeMetrics: [ + { + scope: { name: 'com.anthropic.claude_code', version: '2.1.233' }, + metrics: [ + { + name: 'claude_code.cost.usage', + unit: 'USD', + sum: { + aggregationTemporality: 2, + isMonotonic: true, + dataPoints: [ + { + attributes: kv({ 'session.id': args.sessionId, model: 'claude-haiku-4-5-20251001' }), + timeUnixNano: nanos, + asDouble: 0.0047732, + }, + ], + }, + }, + ], + }, + ], + }, + ], + } +} + +/** + * @param {string} name + * @param {string} timestamp + * @param {Record} attrs + */ +function logRecord(name, timestamp, attrs) { + const nanos = String(BigInt(Date.parse(timestamp)) * 1_000_000n) + return { + timeUnixNano: nanos, + observedTimeUnixNano: nanos, + body: { stringValue: `claude_code.${name}` }, + attributes: kv({ ...attrs, 'event.name': name, 'event.timestamp': timestamp }), + } +} + +/** @param {Record} attrs */ +function kv(attrs) { + return Object.entries(attrs).map(([key, value]) => { + if (typeof value === 'number') { + return Number.isInteger(value) + ? { key, value: { intValue: value } } + : { key, value: { doubleValue: value } } + } + if (typeof value === 'boolean') return { key, value: { boolValue: value } } + return { key, value: { stringValue: String(value) } } + }) +} + +/** + * @param {string} base + * @param {string} sessionId + * @returns {Promise<{ status: number, body: any }>} + */ +async function controlGet(base, sessionId) { + const url = `${base}/_hypaware/ignore/session?${new URLSearchParams({ session_id: sessionId })}` + const res = await fetch(url) + const text = await res.text() + let body + try { + body = JSON.parse(text) + } catch { + body = undefined + } + return { status: res.status, body } +} + +/** + * @param {string} url + * @param {unknown} payload + */ +async function postJson(url, payload) { + const response = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(payload), + }) + // Drain so the socket is released before the listener closes. + await response.text() + return response +} + +/** @param {string} file */ +async function fileExists(file) { + try { + await fs.stat(file) + return true + } catch { + return false + } +} + +/** @param {string} text */ +function streamOf(text) { + return Readable.from([text]) +} + +/** + * @param {{ sql: string, env: any, expect: any, label: string }} args + * @returns {Promise} + */ +async function queryRows({ sql, env, expect, label }) { + const out = makeBuf() + const err = makeBuf() + const code = await dispatch( + ['query', 'sql', sql, '--refresh', 'always', '--format', 'json', '--max-bytes', '0'], + { stdout: out, stderr: err, env } + ) + expect.that(`dispatch: query (${label}) exited 0`, code, (v) => v === 0) + expect.that(`stderr: query (${label}) had no errors`, err.text(), (v) => typeof v === 'string' && v.length === 0) + try { + return JSON.parse(out.text()) + } catch (e) { + expect.that( + `stdout: query (${label}) was valid JSON (${e instanceof Error ? e.message : String(e)})`, + false, + (v) => v === true, + ) + return [] + } +} + +function makeBuf() { + /** @type {string[]} */ + const chunks = [] + return { + /** @param {unknown} chunk */ + write(chunk) { + chunks.push(typeof chunk === 'string' ? chunk : String(chunk)) + return true + }, + text() { + return chunks.join('') + }, + } +} diff --git a/hypaware-core/smoke/flows/client_attach_idempotent.js b/hypaware-core/smoke/flows/client_attach_idempotent.js index 12d9e0ea..d8dd96a1 100644 --- a/hypaware-core/smoke/flows/client_attach_idempotent.js +++ b/hypaware-core/smoke/flows/client_attach_idempotent.js @@ -27,8 +27,12 @@ import { requireAiGatewayRuntime } from '../../plugins-workspace/ai-gateway/src/ * - Detach twice succeeds (second call is a no-op). * - Attach after detach restores the expected state. * - Unrelated user keys survive every attach/detach cycle. - * - Claude attach writes the marker, env.ANTHROPIC_BASE_URL, and the - * managed session-context hooks. + * - Claude attach writes the marker, the LLP 0258 telemetry env block + * (`otel` mode: no base URL, no proxy keys), and the managed + * session-context hooks. + * - Below the Claude Code version floor, attach refuses the switch: + * exit 1, the `claude update` hint on stderr, and the settings file + * byte-identical to before the attempt (LLP 0258 #version-floor). * - Codex attach writes `model_provider = "hypaware"`, the * `[model_providers.hypaware]` table with `base_url`, * `wire_api = "responses"`, and `requires_openai_auth = true`. @@ -86,6 +90,11 @@ export async function run({ harness, expect }) { const previousCodexHome = process.env.CODEX_HOME process.env.HOME = fakeHome process.env.CODEX_HOME = codexHome + // Pin the version the LLP 0258 floor check sees, so the flow never depends + // on whatever `claude` binary the machine running it carries. The refusal + // section below lowers it deliberately, then restores this value. + const previousClaudeVersion = process.env.HYP_CLAUDE_CODE_VERSION + process.env.HYP_CLAUDE_CODE_VERSION = '2.1.233' try { // ---------------------------------------------------------------- @@ -195,6 +204,42 @@ export async function run({ harness, expect }) { afterSecondClaude?._hypaware, (v) => v !== null && typeof v === 'object' && typeof v.port === 'number' ) + // The `otel` attach surface: the telemetry block landed, the marker says + // which mode wrote it and where the spool is, and no routing key exists + // (the Remote Control predicate holds with no override keys). + // @ref LLP 0258#marker-and-spool [tests]: the marker records the mode and the spool directory + expect.that( + 'claude settings: marker records mode=otel with the spool directory', + afterSecondClaude?._hypaware, + (v) => + v !== null && + typeof v === 'object' && + v.mode === 'otel' && + typeof v.spool_dir === 'string' && + v.spool_dir.endsWith(path.join('spool', 'claude-bodies')) + ) + expect.that( + 'claude settings: telemetry env block present (endpoint + spool + enable flag)', + afterSecondClaude?.env, + (v) => + v !== null && + typeof v === 'object' && + v.CLAUDE_CODE_ENABLE_TELEMETRY === '1' && + typeof v.OTEL_EXPORTER_OTLP_ENDPOINT === 'string' && + /^http:\/\/127\.0\.0\.1:\d+$/.test(v.OTEL_EXPORTER_OTLP_ENDPOINT) && + typeof v.OTEL_LOG_RAW_API_BODIES === 'string' && + v.OTEL_LOG_RAW_API_BODIES.startsWith('file:') + ) + expect.that( + 'claude settings: no base URL and no proxy keys were written', + afterSecondClaude?.env, + (v) => + v !== null && + typeof v === 'object' && + !Object.hasOwn(v, 'ANTHROPIC_BASE_URL') && + !Object.hasOwn(v, 'HTTPS_PROXY') && + !Object.hasOwn(v, 'NODE_EXTRA_CA_CERTS') + ) // Attach manages two SessionStart hooks (session-context and the // local-only classify-cwd hook); idempotency means one entry each. expect.that( @@ -236,6 +281,35 @@ export async function run({ harness, expect }) { (v) => typeof v === 'string' && v.includes('No HypAware marker found') ) + // ---------------------------------------------------------------- + // Version floor: below 2.1.193 the switch to `otel` is refused. + // The settings file (back at its seed state here) must not move, + // and the failure must carry the upgrade hint. No fallback to any + // other mode. + // @ref LLP 0258#version-floor [tests]: refusal leaves the settings untouched and prints the `claude update` hint + // ---------------------------------------------------------------- + process.env.HYP_CLAUDE_CODE_VERSION = '2.1.100' + const floorStderr = makeBuf() + code = await runAttach(['--client', 'claude'], { + registry, + kernel, + env: smokeEnv(harness), + stderr: floorStderr, + }) + expect.that('claude attach below the version floor exited 1', code, (v) => v === 1) + expect.that( + 'floor refusal: stderr carries the claude update hint', + floorStderr.text(), + (v) => typeof v === 'string' && v.includes('claude update') + ) + const afterFloorRefusal = await fs.readFile(claudeSettingsPath, 'utf8') + expect.that( + 'floor refusal: settings file is byte-identical to the pre-attempt state', + afterFloorRefusal, + (v) => v === seedClaudeBody + ) + process.env.HYP_CLAUDE_CODE_VERSION = '2.1.233' + code = await runAttach(['--client', 'claude'], { registry, kernel, env }) expect.that('claude attach after detach exited 0', code, (v) => v === 0) const reattachedClaude = await fs.readFile(claudeSettingsPath, 'utf8') @@ -503,10 +577,18 @@ export async function run({ harness, expect }) { claudeAttachSpans, (rows) => rows.every((/** @type {any} */ s) => s.attributes?.hyp_client === 'claude') ) + // One deliberate failure rides this flow now: the version-floor refusal. + // Everything else stays ok, and the refusal is visible as exactly one + // failed adapter span rather than disappearing into the ok count. expect.that( - 'traces: every claude client.attach span has status=ok', + 'traces: exactly one claude client.attach span failed (the floor refusal)', claudeAttachSpans, - (rows) => rows.every((/** @type {any} */ s) => s.attributes?.status === 'ok') + (rows) => rows.filter((/** @type {any} */ s) => s.attributes?.status === 'failed').length === 1 + ) + expect.that( + 'traces: at least 4 claude client.attach spans have status=ok', + claudeAttachSpans, + (rows) => rows.filter((/** @type {any} */ s) => s.attributes?.status === 'ok').length >= 4 ) const codexAttachSpans = traces.filter( @@ -592,6 +674,8 @@ export async function run({ harness, expect }) { else process.env.HOME = previousHome if (previousCodexHome === undefined) delete process.env.CODEX_HOME else process.env.CODEX_HOME = previousCodexHome + if (previousClaudeVersion === undefined) delete process.env.HYP_CLAUDE_CODE_VERSION + else process.env.HYP_CLAUDE_CODE_VERSION = previousClaudeVersion } } diff --git a/hypaware-core/smoke/flows/client_attach_on_join.js b/hypaware-core/smoke/flows/client_attach_on_join.js index 214d3641..17defc5b 100644 --- a/hypaware-core/smoke/flows/client_attach_on_join.js +++ b/hypaware-core/smoke/flows/client_attach_on_join.js @@ -27,8 +27,9 @@ import { dispatch } from '../../../src/core/cli/dispatch.js' * staged restart. * 2. relaunch on rev-1 → first poll clears probation → the confirmation edge * schedules a reconcile pass → **claude auto-attaches**: the `_hypaware` - * marker + the gateway `ANTHROPIC_BASE_URL` land in the client settings, - * and the `attach.claude` client-action marker reads `done`. + * marker + the LLP 0258 telemetry env block land in the client settings + * (`otel` mode: the base URL is never written), and the `attach.claude` + * client-action marker reads `done`. * 3. a second confirmed boot pass (a fresh relaunch on the same rev-1) hits * the **drift** branch of the freshness check: rev-1 lets the gateway bind * an ephemeral port, so the relaunch is at a *new* endpoint, the `done` @@ -73,6 +74,11 @@ export async function run({ harness, expect }) { const previousClaudeHome = process.env.CLAUDE_HOME process.env.HOME = fakeHome delete process.env.CLAUDE_HOME + // Pin the version the LLP 0258 floor check sees: the daemon's auto-attach + // runs the same adapter, and the flow must not depend on whatever `claude` + // binary the machine running it carries. + const previousClaudeVersion = process.env.HYP_CLAUDE_CODE_VERSION + process.env.HYP_CLAUDE_CODE_VERSION = '2.1.233' process.env.HYP_HOME = harness.hypHome delete process.env.HYP_CONFIG @@ -139,11 +145,21 @@ export async function run({ harness, expect }) { attached?._hypaware, (v) => v !== null && typeof v === 'object' && typeof v.port === 'number' ) + // `otel` mode: the telemetry block lands and no routing key is written, + // so the fleet path delivers the same env block a manual attach does. + // @ref LLP 0258#settings-env [tests]: managed settings deliver the same block expect.that( - 'auto-attach: env.ANTHROPIC_BASE_URL points at the local gateway', - attached?.env?.ANTHROPIC_BASE_URL, + 'auto-attach: the telemetry env block points at the loopback listener', + attached?.env?.OTEL_EXPORTER_OTLP_ENDPOINT, (v) => typeof v === 'string' && /^http:\/\/127\.0\.0\.1:\d+$/.test(v) ) + expect.that( + 'auto-attach: mode=otel on the marker, and no base URL was written', + attached, + (v) => + v?._hypaware?.mode === 'otel' && + !Object.hasOwn(v?.env ?? {}, 'ANTHROPIC_BASE_URL') + ) expect.that( 'auto-attach: the unrelated seed key (ANTHROPIC_API_KEY) survived attach', attached?.env?.ANTHROPIC_API_KEY, @@ -171,7 +187,7 @@ export async function run({ harness, expect }) { // ephemeral port, so this boot is at a *different* endpoint: the marker is // stale, the unit is a forward gap, and the attach re-performs at the new // port instead of short-circuiting forever. - // @ref LLP 0086#re-attach-on-drift [tests]: a done marker at a moved endpoint re-performs, which is what keeps ANTHROPIC_BASE_URL pointing at a bound port + // @ref LLP 0086#re-attach-on-drift [tests]: a done marker at a moved endpoint re-performs, which is what keeps the settings marker recording a bound port const driftHandle = await runDaemonHandle(harness) try { await waitFor( @@ -194,10 +210,16 @@ export async function run({ harness, expect }) { (v) => typeof v === 'string' && v.length > 0 && v !== attachedEndpoint ) const rewritten = JSON.parse(await fs.readFile(claudeSettingsPath, 'utf8')) + // In `otel` mode no env key carries the gateway port; the settings + // marker's `port` is what the freshness check compares, so it is what + // must follow the rebound endpoint. expect.that( - 'drift: env.ANTHROPIC_BASE_URL was rewritten to the newly bound port', - rewritten?.env?.ANTHROPIC_BASE_URL, - (v) => typeof v === 'string' && v === drifted?.endpoint + 'drift: the settings marker port was rewritten to the newly bound port', + rewritten?._hypaware?.port, + (v) => + typeof v === 'number' && + typeof drifted?.endpoint === 'string' && + drifted.endpoint.endsWith(`:${v}`) ) expect.that( 'drift: the unrelated seed key (ANTHROPIC_API_KEY) survived the re-attach', @@ -338,9 +360,14 @@ export async function run({ harness, expect }) { (v) => v === undefined ) expect.that( - 'reverse: the managed ANTHROPIC_BASE_URL was removed (no prior to restore)', - JSON.parse(restored)?.env?.ANTHROPIC_BASE_URL, - (v) => v === undefined + 'reverse: the managed telemetry keys were removed (no prior to restore)', + JSON.parse(restored)?.env, + (v) => + v !== null && + typeof v === 'object' && + !Object.hasOwn(v, 'OTEL_EXPORTER_OTLP_ENDPOINT') && + !Object.hasOwn(v, 'CLAUDE_CODE_ENABLE_TELEMETRY') && + !Object.hasOwn(v, 'OTEL_LOG_RAW_API_BODIES') ) expect.that( 'reverse: the unrelated seed key (ANTHROPIC_API_KEY) survived the round-trip', @@ -357,6 +384,8 @@ export async function run({ harness, expect }) { else process.env.HOME = previousHome if (previousClaudeHome === undefined) delete process.env.CLAUDE_HOME else process.env.CLAUDE_HOME = previousClaudeHome + if (previousClaudeVersion === undefined) delete process.env.HYP_CLAUDE_CODE_VERSION + else process.env.HYP_CLAUDE_CODE_VERSION = previousClaudeVersion } await obs.shutdown() diff --git a/hypaware-core/smoke/flows/status_capture_health.js b/hypaware-core/smoke/flows/status_capture_health.js new file mode 100644 index 00000000..fb439659 --- /dev/null +++ b/hypaware-core/smoke/flows/status_capture_health.js @@ -0,0 +1,394 @@ +// @ts-check + +import fs from 'node:fs/promises' +import path from 'node:path' +import process from 'node:process' + +import { installObservability } from '../../../src/core/observability/index.js' +import { dispatch } from '../../../src/core/cli/dispatch.js' +import { createCommandRegistry } from '../../../src/core/registry/commands.js' +import { registerCoreCommands } from '../../../src/core/cli/core_commands.js' +import { createKernelRuntime } from '../../../src/core/runtime/activation.js' +import { activatePlugins } from '../../../src/core/runtime/loader.js' +import { loadManifests } from '../../../src/core/manifest.js' +import { resolveDependencies } from '../../../src/core/dep_graph.js' +import { writeStatusFile } from '../../../src/core/daemon/status.js' + +/** + * Capture-health smoke (LLP 0257 S17, the RFC 0262 open-question-1 duty), + * modeled on `status_diagnostics`. Drives `hyp status` against one + * otel-attached claude install in three states and validates that: + * + * 1. With the listener's `last_event_at` in lockstep with the transcript + * trail, the capture-health line renders, no `capture_gap` diagnostic + * fires, and `overall` stays healthy. + * 2. With transcripts running hours past the last event, `--json` carries + * `capture_health[0].state === 'gap'`, a `capture_gap` diagnostic with an + * `error` severity and a repair hint appears, and `overall` degrades; the + * text surface shows the `[capture gap]` tag. + * 3. With no otel attach marker the line is absent and the `capture_health` + * array is empty - no noise on a machine the question does not apply to. + * + * Everything status reads is a file: the attach marker and transcript mtimes + * under a fake $HOME, and the listener detail under the daemon's status.json + * (no daemon runs; the comparison must survive its daemon, which is exactly + * the down-daemon gap it exists to catch). + * + * @param {{ harness: any, expect: any }} args + */ +export async function run({ harness, expect }) { + const obs = installObservability() + if (!obs.tracer.provider) { + throw new Error( + 'status_capture_health: tracer provider not installed - expected HYP_DEV_TELEMETRY=1' + ) + } + + const cacheRoot = path.join(harness.stateDir, 'cache') + const registry = createCommandRegistry() + registerCoreCommands(registry) + const kernel = createKernelRuntime({ commandRegistry: registry, cacheRoot }) + + const pluginsRoot = path.resolve(import.meta.dirname, '..', '..', 'plugins-workspace') + const pluginDirs = [ + path.join(pluginsRoot, 'ai-gateway'), + path.join(pluginsRoot, 'claude'), + ] + + // Fake $HOME: the attach marker and the transcript trail both live under + // it, and the smoke must never read the developer's real attach state. + const fakeHome = path.join(harness.tmpDir, 'home') + await fs.mkdir(path.join(fakeHome, '.claude'), { recursive: true }) + const previousHome = process.env.HOME + process.env.HOME = fakeHome + + const HOUR = 3_600_000 + const now = Date.now() + + try { + const { loaded } = await loadManifests(pluginDirs) + if (loaded.length !== pluginDirs.length) { + throw new Error( + `status_capture_health: expected ${pluginDirs.length} manifests loaded, got ${loaded.length}` + ) + } + const resolution = await resolveDependencies(loaded.map((l) => l.manifest)) + if (resolution.unsatisfied.length > 0) { + throw new Error( + `status_capture_health: unsatisfied requirements: ${ + resolution.unsatisfied.map((u) => `${u.plugin}:${u.errorKind}`).join(', ') + }` + ) + } + const aiGatewayConfig = { + listen: '127.0.0.1:0', + upstreams: [ + { name: 'anthropic', base_url: 'https://api.anthropic.com', path_prefix: '/' }, + ], + } + const byName = new Map(loaded.map((l) => [l.manifest.name, l])) + const entries = resolution.order + .map((name) => byName.get(name)) + .filter((l) => l !== undefined) + .map((l) => ({ + manifest: l.manifest, + rootDir: l.rootDir, + config: l.manifest.name === '@hypaware/ai-gateway' ? aiGatewayConfig : {}, + })) + await activatePlugins({ + plugins: entries, + stateRoot: harness.stateDir, + runId: harness.devRunId, + runtime: kernel, + tmpRoot: path.join(harness.tmpDir, 'plugin-temp'), + }) + + const configPath = path.join(harness.hypHome, 'hypaware-config.json') + await writeJson(configPath, { + version: 2, + plugins: [ + { + name: '@hypaware/ai-gateway', + config: { + listen: '127.0.0.1:8787', + upstreams: [ + { name: 'anthropic', base_url: 'https://api.anthropic.com', path_prefix: '/' }, + ], + }, + }, + { name: '@hypaware/claude', config: { proxy: '@hypaware/ai-gateway' } }, + ], + }) + + // The otel attach marker a real `hyp attach --client claude` writes, + // stubbed the way status_diagnostics stubs the base-URL one so the + // assertions focus on the health surface rather than adapter effects. + const attachedAt = new Date(now - 6 * HOUR).toISOString() + await writeJson(path.join(fakeHome, '.claude', 'settings.json'), { + _hypaware: { + attached_at: attachedAt, + version: '2.0.0', + port: 8787, + mode: 'otel', + spool_dir: path.join(harness.hypHome, 'spool', 'claude-bodies'), + managed: { env: {}, hooks: [] }, + }, + env: {}, + }) + + // One transcript, freshly written: the client-side half of the + // comparison. + const transcript = path.join(fakeHome, '.claude', 'projects', '-tmp-proj', 'aaaa-session.jsonl') + await fs.mkdir(path.dirname(transcript), { recursive: true }) + await fs.writeFile(transcript, '{}\n') + const transcriptMtime = new Date(now - 30_000) + await fs.utimes(transcript, transcriptMtime, transcriptMtime) + + /* ---------- Case 1: events in lockstep -> line, no gap, healthy ---------- */ + + writeListenerStatus(harness.stateDir, new Date(now - 60_000).toISOString()) + + const okStdout = makeBuf() + const okExit = await dispatch(['status'], { + stdout: okStdout, + stderr: makeBuf(), + kernel, + registry, + env: smokeEnv({ harness, hypConfig: configPath }), + }) + expect.that('healthy: hyp status exited 0', okExit, (v) => v === 0) + const okText = okStdout.text() + expect.that( + 'healthy: capture-health line renders', + okText, + (v) => v.includes('capture health:') && /- claude {2}last event .*, last transcript activity /.test(v) + ) + expect.that( + 'healthy: no capture-gap tag', + okText, + (v) => !v.includes('[capture gap]') + ) + expect.that( + 'healthy: overall stays healthy', + okText, + (v) => v.includes('overall: healthy') + ) + + /* ---------- Case 2: transcripts hours past the last event -> degraded ---------- */ + + writeListenerStatus(harness.stateDir, new Date(now - 5 * HOUR).toISOString()) + + const gapStdout = makeBuf() + const gapExit = await dispatch(['status', '--json'], { + stdout: gapStdout, + stderr: makeBuf(), + kernel, + registry, + env: smokeEnv({ harness, hypConfig: configPath }), + }) + expect.that('gap json: hyp status --json exited 0', gapExit, (v) => v === 0) + /** @type {any} */ + let gapJson + try { + gapJson = JSON.parse(gapStdout.text()) + } catch (err) { + const message = err instanceof Error ? err.message : String(err) + expect.that(`gap json: parseable (${message})`, false, (v) => v === true) + } + expect.that( + 'gap json: capture_health carries the gap entry', + gapJson?.capture_health, + (v) => Array.isArray(v) && v.length === 1 && v[0].client === 'claude' && + v[0].state === 'gap' && typeof v[0].gap_seconds === 'number' && v[0].gap_seconds > 4 * 3600 + ) + expect.that( + 'gap json: client_attach reports the otel mode', + gapJson?.client_attach, + (v) => Array.isArray(v) && v.some( + (/** @type {any} */ c) => c.name === 'claude' && c.attached === true && c.mode === 'otel' + ) + ) + const gapDiag = (gapJson?.diagnostics ?? []).find( + (/** @type {any} */ d) => d.kind === 'capture_gap' + ) + expect.that( + 'gap json: capture_gap diagnostic fires at error severity', + gapDiag, + (v) => v !== undefined && v.severity === 'error' && typeof v.message === 'string' + ) + expect.that( + 'gap json: the diagnostic carries a repair hint', + gapDiag?.repair, + (v) => Array.isArray(v) && v.length > 0 && v.some( + (/** @type {any} */ r) => typeof r === 'string' && r.includes('hyp attach --client claude') + ) + ) + expect.that( + 'gap json: overall degrades', + gapJson?.overall, + (v) => v === 'degraded' + ) + + const gapTextStdout = makeBuf() + const gapTextExit = await dispatch(['status'], { + stdout: gapTextStdout, + stderr: makeBuf(), + kernel, + registry, + env: smokeEnv({ harness, hypConfig: configPath }), + }) + expect.that('gap text: hyp status exited 0', gapTextExit, (v) => v === 0) + const gapText = gapTextStdout.text() + expect.that( + 'gap text: the line carries the capture-gap tag', + gapText, + (v) => /- claude {2}last event .*, last transcript activity .* {2}\[capture gap\]/.test(v) + ) + expect.that( + 'gap text: diagnostics name capture_gap', + gapText, + (v) => v.includes('capture_gap') + ) + + /* ---------- Case 3: no otel attach -> no line, no noise ---------- */ + + await fs.rm(path.join(fakeHome, '.claude', 'settings.json'), { force: true }) + + const offStdout = makeBuf() + const offExit = await dispatch(['status', '--json'], { + stdout: offStdout, + stderr: makeBuf(), + kernel, + registry, + env: smokeEnv({ harness, hypConfig: configPath }), + }) + expect.that('detached json: hyp status --json exited 0', offExit, (v) => v === 0) + /** @type {any} */ + let offJson + try { + offJson = JSON.parse(offStdout.text()) + } catch (err) { + const message = err instanceof Error ? err.message : String(err) + expect.that(`detached json: parseable (${message})`, false, (v) => v === true) + } + expect.that( + 'detached json: capture_health is empty', + offJson?.capture_health, + (v) => Array.isArray(v) && v.length === 0 + ) + expect.that( + 'detached json: no capture_gap diagnostic', + (offJson?.diagnostics ?? []).map((/** @type {any} */ d) => d.kind), + (v) => Array.isArray(v) && !v.includes('capture_gap') + ) + + const offTextStdout = makeBuf() + await dispatch(['status'], { + stdout: offTextStdout, + stderr: makeBuf(), + kernel, + registry, + env: smokeEnv({ harness, hypConfig: configPath }), + }) + expect.that( + 'detached text: no capture-health section', + offTextStdout.text(), + (v) => !v.includes('capture health') + ) + + await obs.shutdown() + + /* ---------- Span assertions ---------- */ + + const traces = await expect.traces() + const statusSpans = traces.filter( + (/** @type {any} */ t) => t.name === 'status.render' + ) + expect.that( + 'traces: five status.render spans (healthy + gap json + gap text + detached json + detached text)', + statusSpans, + (v) => Array.isArray(v) && v.length >= 5 + ) + const degradedSpan = statusSpans.find( + (/** @type {any} */ s) => s.attributes?.overall === 'degraded' + ) + expect.that( + 'traces: the gap run records degraded with a diagnostic counted', + degradedSpan?.attributes, + (v) => v !== undefined && v.diagnostics_count >= 1 + ) + const healthySpan = statusSpans.find( + (/** @type {any} */ s) => s.attributes?.overall === 'healthy' + ) + expect.that( + 'traces: the lockstep run stayed healthy', + healthySpan, + (v) => v !== undefined + ) + } finally { + if (previousHome === undefined) delete process.env.HOME + else process.env.HOME = previousHome + } +} + +/** + * The listener detail the daemon tick would have collected into + * status.json: the claude-telemetry snapshot with `last_event_at` + * (LLP 0257 S16). No pid file rides beside it on purpose - the + * comparison is not liveness-gated. + * + * @param {string} stateRoot + * @param {string | null} lastEventAt + */ +function writeListenerStatus(stateRoot, lastEventAt) { + writeStatusFile(stateRoot, /** @type {any} */ ({ + state: 'healthy', + sources: [ + { + name: 'ai-gateway', + plugin: '@hypaware/ai-gateway', + state: 'started', + details: { host: '127.0.0.1', port: 8787 }, + }, + { + name: 'claude-telemetry', + plugin: '@hypaware/claude', + state: 'started', + details: { listen_host: '127.0.0.1', listen_port: 4319, last_event_at: lastEventAt }, + }, + ], + sinks: [], + })) +} + +/** + * @param {{ harness: { hypHome: string }, hypConfig: string }} args + */ +function smokeEnv({ harness, hypConfig }) { + return { ...process.env, HYP_HOME: harness.hypHome, HYP_CONFIG: hypConfig } +} + +/** + * @param {string} filePath + * @param {unknown} value + */ +async function writeJson(filePath, value) { + await fs.mkdir(path.dirname(filePath), { recursive: true }) + await fs.writeFile(filePath, JSON.stringify(value, null, 2) + '\n', 'utf8') +} + +function makeBuf() { + /** @type {string[]} */ + const chunks = [] + return { + chunks, + /** @param {unknown} chunk */ + write(chunk) { + chunks.push(typeof chunk === 'string' ? chunk : String(chunk)) + return true + }, + text() { + return chunks.join('') + }, + } +} diff --git a/hypaware-core/smoke/flows/walkthrough_picker_to_first_query.js b/hypaware-core/smoke/flows/walkthrough_picker_to_first_query.js index 989e2162..35a880d4 100644 --- a/hypaware-core/smoke/flows/walkthrough_picker_to_first_query.js +++ b/hypaware-core/smoke/flows/walkthrough_picker_to_first_query.js @@ -104,6 +104,10 @@ export async function run({ harness, expect }) { await fs.mkdir(path.join(fakeHome, '.codex'), { recursive: true }) const previousHome = process.env.HOME process.env.HOME = fakeHome + // Pin the version the LLP 0258 floor check sees, so the init-driven attach + // never depends on whatever `claude` binary the machine running it carries. + const previousClaudeVersion = process.env.HYP_CLAUDE_CODE_VERSION + process.env.HYP_CLAUDE_CODE_VERSION = '2.1.233' // Pre-existing settings files would let us detect that dry-runs do // not modify them. Seed harmless baselines and snapshot them. @@ -430,10 +434,20 @@ export async function run({ harness, expect }) { realClaudeSettings?._hypaware?.port, (v) => v === 18521 ) + // `otel` attach (LLP 0258): the telemetry block is written and the base + // URL is not; the marker's port above is what carries the gateway + // endpoint for the drift check. expect.that( - 'real init attach: claude base URL uses the default gateway endpoint', - realClaudeSettings?.env?.ANTHROPIC_BASE_URL, - (v) => v === 'http://127.0.0.1:18521' + 'real init attach: the telemetry endpoint points at the loopback listener', + realClaudeSettings?.env?.OTEL_EXPORTER_OTLP_ENDPOINT, + (v) => typeof v === 'string' && /^http:\/\/127\.0\.0\.1:\d+$/.test(v) + ) + expect.that( + 'real init attach: no base URL was written (mode=otel)', + realClaudeSettings, + (v) => + v?._hypaware?.mode === 'otel' && + !Object.hasOwn(v?.env ?? {}, 'ANTHROPIC_BASE_URL') ) // ----- 7. Span + log assertions ----- @@ -548,6 +562,8 @@ export async function run({ harness, expect }) { } finally { if (previousHome === undefined) delete process.env.HOME else process.env.HOME = previousHome + if (previousClaudeVersion === undefined) delete process.env.HYP_CLAUDE_CODE_VERSION + else process.env.HYP_CLAUDE_CODE_VERSION = previousClaudeVersion await echo.close() } } diff --git a/hypaware-plugin-kernel-types.d.ts b/hypaware-plugin-kernel-types.d.ts index 3b8bf4ad..4a7d2ebc 100644 --- a/hypaware-plugin-kernel-types.d.ts +++ b/hypaware-plugin-kernel-types.d.ts @@ -202,6 +202,29 @@ export interface PluginClientManifest { * attachable but never launchable. */ launch?: PluginClientLaunchManifest + /** + * Where this client's own activity leaves a file trail, for the + * `hyp status` capture-health comparison (LLP 0257#status-and-health): + * the newest matching mtime under `dir` is the client's last activity, + * which status holds against the telemetry the daemon actually + * captured. Declared here rather than in a core table for the same + * reason as `attach_probe`: the path is the client's business, and + * core must be able to read it without importing plugin code. + */ + activity_probe?: PluginActivityProbeManifest +} + +/** + * A client-written directory core may stat (never parse) to answer + * "when was this client last active?". Same home-relative contract as + * `attach_probe.settings_file`: relative to `$HOME`, first segment + * relocatable by `$_HOME`, absolute paths rejected. + */ +export interface PluginActivityProbeManifest { + /** Directory of activity files, RELATIVE to the user's home (e.g. `.claude/projects`). */ + dir: string + /** Only files ending in this suffix count (e.g. `.jsonl`); absent means every file. */ + file_suffix?: string } /** @@ -1729,6 +1752,22 @@ export interface AiGatewayCapability { * LLP 0024. */ registerSettlementEnricher(enricher: AiGatewaySettlementEnricher): void + /** + * Record one already-projected exchange into `ai_gateway_messages`. + * + * For a LIVE producer that does not sit on the wire: it holds a + * finished `AiGatewayProjectedExchange` and hands it to the dataset's + * owner rather than learning the table path, the column list, and the + * `part_id` dedupe rules. The proxy recorder does not use this (it + * already owns the projector chain); the Claude OTEL telemetry + * listener does. Rows whose `part_id` another producer already stored + * are skipped, which is what makes producer overlap harmless. See + * LLP 0252 #projection-unchanged. + */ + recordProjectedExchange( + exchange: AiGatewayProjectedExchange, + opts?: AiGatewayRecordOptions, + ): Promise localEndpoint(opts?: AiGatewayEndpointOptions): string /** * Look up a registered client by name. Returns `undefined` when no @@ -1744,6 +1783,24 @@ export interface AiGatewayCapability { listClients(): AiGatewayClientRegistration[] } +/** Options for `AiGatewayCapability.recordProjectedExchange`. */ +export interface AiGatewayRecordOptions { + /** + * Producer provenance merged under every emitted row's `attributes`, + * the same slot the backfill materializer fills with + * `{ gateway: { source: 'backfill' } }`. + */ + gatewayAttributes?: JsonObject +} + +/** Outcome of one `recordProjectedExchange` call. */ +export interface AiGatewayRecordResult { + /** Rows appended to the dataset. */ + rowsWritten: number + /** Rows dropped because another producer already stored that `part_id`. */ + rowsSkipped: number +} + /** * Adapter-contributed flush-time enricher. Given the selected rows of a * flush batch (already filtered to this enricher's `clientName`), it diff --git a/llp/0085-settlement-may-drop-late-ignore.decision.md b/llp/0085-settlement-may-drop-late-ignore.decision.md index 5046b66e..ccd49f37 100644 --- a/llp/0085-settlement-may-drop-late-ignore.decision.md +++ b/llp/0085-settlement-may-drop-late-ignore.decision.md @@ -7,6 +7,10 @@ **Author:** Phil / Claude **Date:** 2026-07-07 **Related:** LLP 0027, LLP 0049, LLP 0050, LLP 0070, LLP 0083 +**Extended-by:** LLP 0254 (#scope: accepted 2026-08-17; the Claude OTEL +listener resolves the policy at ingest with cwd already in hand, so the race +this backstop exists for does not arise there and the late drop does not run on +that path; it stands unchanged for the live proxy and for transcript backfill) > When a Claude exchange raced past the capture seam with `cwd = null` (the > session-start hook record had not landed yet), the flush-time settlement diff --git a/llp/0104-hyp-purge.decision.md b/llp/0104-hyp-purge.decision.md index 727b9ea6..70197297 100644 --- a/llp/0104-hyp-purge.decision.md +++ b/llp/0104-hyp-purge.decision.md @@ -6,6 +6,9 @@ **Author:** Phil / Claude **Date:** 2026-07-13 **Related:** LLP 0049, LLP 0030, LLP 0050, LLP 0069, LLP 0100, LLP 0103 +**Extended-by:** LLP 0253 (#purge-and-detach-sweep: accepted 2026-08-17; every form of the verb also empties the raw-body capture spool, which holds +un-projected bodies rather than cached rows; the target shapes, the +confirmation gate, and the cache-only stance here are unchanged) > Retroactive deletion arrives as its own destructive verb. `hyp purge` > removes already-cached rows by subtree, by session, by resolved-`ignore` diff --git a/llp/0231-proxy-mode-capture.rfc.md b/llp/0231-proxy-mode-capture.rfc.md index 8fe82706..d6f2d3d5 100644 --- a/llp/0231-proxy-mode-capture.rfc.md +++ b/llp/0231-proxy-mode-capture.rfc.md @@ -7,6 +7,10 @@ **Date:** 2026-08-14 **Related:** LLP 0016, LLP 0044, LLP 0045, LLP 0049, LLP 0066, LLP 0086, LLP 0114, LLP 0116, LLP 0176, LLP 0192, LLP 0206 **Spawns:** LLP 0232, LLP 0233, LLP 0234, LLP 0235 +**Extended-by:** LLP 0262 (accepted 2026-08-17; the `claude` client stops +being captured by proxy at all and rides Claude Code's own telemetry export +instead; the aperture reasoning and the proxy itself stay in force for every +other client) > Claude Code disables **Remote Control** whenever `ANTHROPIC_BASE_URL` points > anywhere other than `api.anthropic.com`. Attach repoints exactly that key, so diff --git a/llp/0232-claude-attaches-by-proxy.decision.md b/llp/0232-claude-attaches-by-proxy.decision.md index 6c15128e..921d4215 100644 --- a/llp/0232-claude-attaches-by-proxy.decision.md +++ b/llp/0232-claude-attaches-by-proxy.decision.md @@ -13,6 +13,9 @@ see LLP 0236); LLP 0247 (#attach-writes-https_proxy-not-a-base-url: the gateway now serves absolute-form request-targets to registered hosts on forward-proxy listeners, so the case against `HTTP_PROXY` rests on the no-plaintext-traffic-worth-capturing rationale alone) +**Superseded-by (in part):** LLP 0262, LLP 0258 (accepted 2026-08-17; attaching the `claude` client writes a telemetry `env` block instead of +`HTTPS_PROXY` and `NODE_EXTRA_CA_CERTS`; #mode-migration and the `prev_env` +undo record are what the third mode reuses unchanged) > Attach stops writing `env.ANTHROPIC_BASE_URL` and writes `env.HTTPS_PROXY` > plus `env.NODE_EXTRA_CA_CERTS` instead. The endpoint stays diff --git a/llp/0235-local-ca-lifecycle.decision.md b/llp/0235-local-ca-lifecycle.decision.md index fe7e6e97..e5cbde99 100644 --- a/llp/0235-local-ca-lifecycle.decision.md +++ b/llp/0235-local-ca-lifecycle.decision.md @@ -9,6 +9,9 @@ **Superseded-by (in part):** LLP 0237 (#client-scoped-trust), LLP 0238 (#detach-removes-the-ca, #ca-name-constraints mint-from-routing-table, #ca-lifecycle one-year validity) +**Extended-by:** LLP 0262 (accepted 2026-08-17; the `claude` client's +attach no longer depends on the CA; the CA lifecycle stands for any client +still routed through the proxy) > The machine-local certificate authority is generated in-process with no > `openssl` shell-out and no new dependency, constrained to the hosts it diff --git a/llp/0237-attach-trusts-ca-in-login-keychain.decision.md b/llp/0237-attach-trusts-ca-in-login-keychain.decision.md index 61bc3a76..f881959e 100644 --- a/llp/0237-attach-trusts-ca-in-login-keychain.decision.md +++ b/llp/0237-attach-trusts-ca-in-login-keychain.decision.md @@ -6,6 +6,10 @@ **Author:** Phil / Claude **Date:** 2026-08-15 **Related:** LLP 0044, LLP 0232, LLP 0235, LLP 0236, LLP 0238, LLP 0239 +**Extended-by:** LLP 0262 (accepted 2026-08-17; attaching the `claude` +client asks for no keychain trust at all; this decision governs the clients +still proxied, and `detach --purge` stays the removal path for a grant a +migrated machine already made) > On macOS, proxy-mode attach installs the interception CA into the user's > login keychain as a user-domain trusted root, via `security diff --git a/llp/0238-long-lived-ca-full-provider-constraints.decision.md b/llp/0238-long-lived-ca-full-provider-constraints.decision.md index cd0c187b..26c641bb 100644 --- a/llp/0238-long-lived-ca-full-provider-constraints.decision.md +++ b/llp/0238-long-lived-ca-full-provider-constraints.decision.md @@ -6,6 +6,9 @@ **Author:** Phil / Claude **Date:** 2026-08-15 **Related:** LLP 0234, LLP 0235, LLP 0236, LLP 0237 +**Extended-by:** LLP 0262 (accepted 2026-08-17; the long-lived CA is not +part of capturing the `claude` client; it remains the credential for any +client still routed through the proxy) > The interception CA becomes a per-machine, ten-year credential whose name > constraints permit every provider host HypAware can ever intercept, so the diff --git a/llp/0239-node-use-system-ca-via-launchd.decision.md b/llp/0239-node-use-system-ca-via-launchd.decision.md index 6227b123..03714014 100644 --- a/llp/0239-node-use-system-ca-via-launchd.decision.md +++ b/llp/0239-node-use-system-ca-via-launchd.decision.md @@ -6,6 +6,10 @@ **Author:** Phil / Claude **Date:** 2026-08-15 **Related:** LLP 0232, LLP 0236, LLP 0237, LLP 0238 +**Extended-by:** LLP 0262, LLP 0258 (accepted 2026-08-17; the OTEL attach +delivers its environment through the settings `env` block, which reaches +every session with no launchd write and no terminal restart, so this +delivery mechanism is not used for the `claude` client) > Proxy-mode attach sets `NODE_USE_SYSTEM_CA=1` with `launchctl setenv` and > installs a LaunchAgent that re-applies it at login, because the variable diff --git a/llp/0242-fresh-installs-attach-base-url.issue.md b/llp/0242-fresh-installs-attach-base-url.issue.md index f0ee2d2b..137ff93c 100644 --- a/llp/0242-fresh-installs-attach-base-url.issue.md +++ b/llp/0242-fresh-installs-attach-base-url.issue.md @@ -6,6 +6,9 @@ **Author:** Phil / Claude **Date:** 2026-08-17 **Related:** LLP 0231, LLP 0232, LLP 0233, LLP 0174 +**Extended-by:** LLP 0262 (accepted 2026-08-17; the resolution below is +itself migrated: the `claude` client's attach target becomes the OTEL mode, +so proxy mode stops being what fresh installs compose for it) > Proxy-mode capture shipped (LLP 0231-0239) but nothing writes > `proxy_mode: true`, so every install path still lands on the base-URL diff --git a/llp/0243-picker-composes-proxy-mode.decision.md b/llp/0243-picker-composes-proxy-mode.decision.md index d88cff6e..c0e5967a 100644 --- a/llp/0243-picker-composes-proxy-mode.decision.md +++ b/llp/0243-picker-composes-proxy-mode.decision.md @@ -8,6 +8,9 @@ **Related:** LLP 0130, LLP 0135, LLP 0213, LLP 0232, LLP 0242 **Extends:** LLP 0233 (#proxy-mode-is-explicit: the key is still the only switch and is still explicit in the file; what changes is who writes it) +**Extended-by:** LLP 0262 (accepted 2026-08-17; the Claude row stops +declaring `compose.gateway_proxy_mode`, because the client it composes for +is no longer captured by proxy; the composition rule itself is unchanged) > A picker row that attaches its client by proxy declares > `compose.gateway_proxy_mode: true` in its manifest. The composition fold diff --git a/llp/0244-attach-migrates-to-proxy-mode.decision.md b/llp/0244-attach-migrates-to-proxy-mode.decision.md index 955da846..4af4c5fb 100644 --- a/llp/0244-attach-migrates-to-proxy-mode.decision.md +++ b/llp/0244-attach-migrates-to-proxy-mode.decision.md @@ -7,6 +7,10 @@ **Date:** 2026-08-17 **Related:** LLP 0031, LLP 0174, LLP 0181, LLP 0232, LLP 0233, LLP 0242, LLP 0243 +**Extended-by:** LLP 0262 (accepted 2026-08-17; the migration machinery +here is retargeted, not reverted: `hyp attach claude` migrates a proxy-mode +install to the OTEL mode, unwinding the launchd environment and offering +`detach --purge` for the CA trust) > When `hyp attach claude` runs against an effective config whose gateway > block lacks `proxy_mode: true`, and the client's row declares proxy attach, diff --git a/llp/0252-events-carry-content-bodies-fill-the-gaps.decision.md b/llp/0252-events-carry-content-bodies-fill-the-gaps.decision.md new file mode 100644 index 00000000..ade35cbb --- /dev/null +++ b/llp/0252-events-carry-content-bodies-fill-the-gaps.decision.md @@ -0,0 +1,73 @@ +# LLP 0252: Events carry the content, body files fill the gaps, and the body is deleted + +**Type:** Decision +**Status:** Accepted +**Systems:** Sources, Plugins, Privacy +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0012, LLP 0016, LLP 0030, LLP 0032, LLP 0262 (the RFC this +decision realizes, accepted 2026-08-17), LLP 0253, LLP 0257, LLP 0258 + +> The OTEL event stream is the primary producer: it arrives pre-deduplicated +> and supplies identity, content, usage, and behavioral data. Raw body files +> are read only for what events do not carry (system text, the tools list, +> message ordering, untruncated tool args), and each body file is deleted as +> soon as it has been projected. + +## Context + +Claude Code exports two things at once: an event stream over OTLP, and raw +request and response bodies written to a directory. Either alone is incomplete +(LLP 0262 records the measurements). This decision settles how the two are +combined, and what happens to a body file afterwards. + +## Decision + +### Events are the spine {#events-first} + +**Each piece of content is taken from the event that carries it exactly once.** +`user_prompt`, `assistant_response`, and `tool_result` are emitted once per +occurrence, so the stream is naturally incremental and needs no windowing, +no replay, and no settlement pass to decide what is new. `message.uuid` on the +event is the row identity. + +### Bodies are consulted, not ingested wholesale {#bodies-for-gaps} + +**A body file is read for the fields events lack and for nothing else**: +`system_text`, the `tools` list, message ordering, and untruncated tool +arguments (event `tool_input` clips values at 512 characters). The body is +located through `api_request_body.body_ref`. Ingesting bodies as the primary +content source would re-import the whole message history every turn and put the +part-level dedupe back on the hot path for content the events already delivered +once. + +### Projected, then deleted {#project-then-delete} + +**A body file is deleted immediately after it is projected**, successfully or +not: a body that cannot be projected is not retried forever, because the same +session is recoverable from transcript backfill and an undeleted body is a raw +prompt sitting on disk. Deletion is the normal end of a body's life, not a +cleanup pass, which is what keeps the spool transient rather than an archive. + +### OTEL is a third producer, not a new table {#projection-unchanged} + +**The listener yields the same `ai_gateway.projected_exchange` values the live +proxy and the backfill providers yield today.** The `ai_gateway_messages` +dataset, its `part_id` dedupe, its partitioning (LLP 0030), and its repo +identity columns (LLP 0032) are untouched. The overlap window during migration +is therefore harmless: two producers writing the same parts dedupe into one +row. + +## Consequences + +- Every existing query, report, and graph consumer of `ai_gateway_messages` + keeps working with no change. +- A session captured while the daemon was down loses its events but keeps its + bodies until the spool cap evicts them (LLP 0253); what neither survives, + transcript backfill recovers. +- `parent_uuid`, `logical_parent_uuid`, `user_type`, and `permission_mode` + read null on this path. They stay in the schema; `query_source` and + `agent.name` are the attribution source for sidechain and agent identity. +- A body-format change upstream degrades exactly one axis (system text, tools, + ordering, long tool args) instead of stopping capture, because the events + still carry the content. diff --git a/llp/0253-body-spool-is-capped-and-swept.decision.md b/llp/0253-body-spool-is-capped-and-swept.decision.md new file mode 100644 index 00000000..49a3f2b2 --- /dev/null +++ b/llp/0253-body-spool-is-capped-and-swept.decision.md @@ -0,0 +1,80 @@ +# LLP 0253: The body spool is owner-only, byte-capped, evicted oldest-first, and swept on removal + +**Type:** Decision +**Status:** Accepted +**Systems:** Privacy, Config, Sources, Daemon +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0049, LLP 0066, LLP 0085, LLP 0103, LLP 0262 (the RFC this +decision realizes, accepted 2026-08-17), LLP 0252, LLP 0257, LLP 0258 +**Extended-by:** LLP 0263 (#byte-cap is also enforced by the client hook, so +the bound holds while the daemon is down) + +> Raw request and response bodies land in a spool directory under the HypAware +> home with owner-only permissions. Its size is a config value with a 512 MB +> default and oldest-first eviction, so a down daemon can never fill the disk. +> Bodies belonging to an ignored or policy-dropped session are deleted, not +> skipped, and `hyp purge` and detach both sweep the directory. + +## Context + +Claude Code writes bodies to a directory we name at attach (LLP 0258 +#env-keys) and it keeps writing whether or not the daemon is reading. Measured +volume is about 145 KB per request, so a heavy day passes gigabytes through the +directory. LLP 0262 settles that transient spool presence is acceptable, given +the same content already sits in `~/.claude/projects`, on three conditions. +This decision is those conditions. + +## Decision + +### The spool lives under the HypAware home, owner-only {#spool-location} + +**The directory is `/spool/claude-bodies`, created mode `0700`.** +Raw prompts must not be world-readable, and a path under the HypAware home is +one the user already knows to be ours: it is what `hyp purge` and detach can +find without being told, and what a backup tool that excludes the HypAware home +already excludes. + +### A byte cap with oldest-first eviction {#byte-cap} + +**The spool is bounded by a configured byte cap, default 512 MB, and the oldest +files are removed first when it is exceeded.** The cap is enforced by the +daemon, not by hoping the reader keeps up: the window this exists for is +precisely the one where the reader is not running. Oldest-first is the right +direction because the newest bodies are the ones whose events are still +arriving. + +### Eviction degrades to backfill, never to loss {#eviction-degrades} + +**An evicted body is not an error.** The content it held is recoverable from +the Claude Code transcript by the existing backfill path, so the failure mode +of a full spool is "captured later, with less detail", not "captured never" and +not "disk full". Eviction is logged with a count so a machine that is +routinely evicting is visible. + +### Dropped sessions have their bodies deleted {#delete-on-drop} + +**When ingest drops a session (`.hypignore`, the machine-local list of LLP 0049 +and 0103, or a per-session ignore under LLP 0066), it deletes that session's +bodies instead of leaving them unread.** Skipping would leave the content of +exactly the sessions the user asked us not to keep sitting in our own +directory until a cap evicted it. Deletion is what makes the opt-out mean what +it says. + +### Purge and detach sweep the spool {#purge-and-detach-sweep} + +**`hyp purge` and `hyp detach claude` both remove the spool directory's +contents.** The attach marker records the path (LLP 0258 #marker-and-spool) so +neither verb has to recompute it, and so a detach after a config change still +sweeps the directory that was actually used. + +## Consequences + +- Disk growth from capture is bounded by the cap plus the Iceberg cache, and + the cap is one config value an operator can lower on a small disk. +- A stopped daemon costs detail, not integrity, and the loss has a named + recovery path. +- The spool is a privacy surface with three named duties, so a review can check + it against this list rather than against intent. +- Nothing in the spool outlives the user's decision to remove it, which is what + lets LLP 0262 accept transient presence at all. diff --git a/llp/0254-otel-path-settles-at-ingest.decision.md b/llp/0254-otel-path-settles-at-ingest.decision.md new file mode 100644 index 00000000..5185e87d --- /dev/null +++ b/llp/0254-otel-path-settles-at-ingest.decision.md @@ -0,0 +1,65 @@ +# LLP 0254: The OTEL path settles at ingest, so flush-time settlement and its late drop are not used + +**Type:** Decision +**Status:** Accepted +**Systems:** Gateway, Cache, Plugins, Privacy +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0027, LLP 0049, LLP 0050, LLP 0085, LLP 0103, LLP 0262 (the +RFC this decision realizes, accepted 2026-08-17), LLP 0252, LLP 0257 + +> Events carry `message.uuid`, so a row's identity is known when it is written +> and there is no provisional row to settle later. The usage-policy check runs +> inline at ingest with cwd in hand, so there is no late-resolved drop either. +> Both mechanisms stay exactly as they are for the proxy and backfill paths. + +## Context + +LLP 0027 exists because the proxy sees an exchange before the session's +identity is known, so rows are written provisionally and repaired at flush. +LLP 0085 exists because that repair window let a row whose `.hypignore` verdict +resolved late slip through. Neither cause is present when the producer stamps +identity on every event. + +## Decision + +### Native identity ends the settlement race {#identity-at-ingest} + +**A row projected from an OTEL event is final when written.** `session.id`, +`message.uuid`, `prompt.id`, and `request_id` arrive on the event itself, so +nothing is provisional and the flush-time settlement pass of LLP 0027 has +nothing to repair on this path. + +### The policy check runs inline, with cwd known {#policy-inline} + +**`.hypignore` and the machine-local list (LLP 0049, LLP 0103) are evaluated at +ingest, before the row is written**, using the cwd the retained SessionStart +hook recorded and the existing usage-policy drop sentinel. A row that must not +exist is never written, rather than written and dropped later, so the fail-open +window LLP 0085 patches cannot reappear here: there is no window. + +### The SessionStart hook stays {#hook-stays} + +**The hook remains the source of `cwd`, `git_branch`, `git_remote`, +`head_sha`, and `repo_root`.** Events do not carry `workspace.host_paths` on a +plain local session (LLP 0262 spike finding), and deriving repo identity from +the body's system text is parseable but fragile. Removing the hook would cost +both the repo columns and the inline policy check that depends on them. + +### Scope: this path only {#scope} + +**LLP 0027 and LLP 0085 remain in force for the live proxy and for transcript +backfill.** They are not retired, superseded, or deleted; those producers still +write provisional rows and still need the late drop. This decision narrows +where the machinery runs, and nothing else. + +## Consequences + +- One less asynchronous repair stage on the hot capture path, and one less + place a privacy verdict can arrive after the data does. +- The hook is now load-bearing for privacy on this path, not only for repo + identity, so a session with no hook record has no cwd and must be treated as + undetermined rather than as clean. +- Transcript backfill keeps its own settlement behavior, so a machine running + both producers has both regimes live at once, which is expected and already + the case today. diff --git a/llp/0255-claude-telemetry-events-dataset.decision.md b/llp/0255-claude-telemetry-events-dataset.decision.md new file mode 100644 index 00000000..7dd58a3d --- /dev/null +++ b/llp/0255-claude-telemetry-events-dataset.decision.md @@ -0,0 +1,70 @@ +# LLP 0255: Behavioral telemetry lands in its own `claude_telemetry_events` dataset + +**Type:** Decision +**Status:** Accepted +**Systems:** Plugins, Query, Sources +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0012, LLP 0014, LLP 0015, LLP 0016, LLP 0030, LLP 0262 (the +RFC this decision realizes, accepted 2026-08-17), LLP 0252, LLP 0257 + +> The events that describe behavior rather than conversation (tool accept and +> reject decisions, permission mode changes, hook executions, MCP server +> health, cost and activity metrics) get their own dataset owned by +> `@hypaware/claude`: one row per event, typed columns for the hot fields, an +> attributes JSON column for the rest. + +## Context + +The OTEL stream carries two different kinds of thing. One is the conversation, +which already has a home in `ai_gateway_messages`. The other is behavior the +wire never showed, and it has no home at all. LLP 0262 resolved that it needs +one; this decision settles which one. + +## Decision + +### A dataset of its own {#own-dataset} + +**`claude_telemetry_events` is a new dataset, not a widening of +`ai_gateway_messages` and not a route through `@hypaware/otel`'s generic +`logs` / `metrics` datasets.** Widening the message table would add columns +that are null for every row from every other producer and for most rows from +this one. Routing through the generic OTEL datasets would put Claude-specific +attributes behind a shape whose columns describe OTLP, not Claude Code, so +every question would be asked through JSON extraction. + +### One row per event, hot fields typed {#row-shape} + +**Each event becomes one row.** Typed columns cover the fields queries filter +and group by (event name, session id, tool name, decision, decision source, +cost); everything else rides in an `attributes` JSON column. The split is a +query-ergonomics judgment, not a completeness one: no attribute is dropped, and +a field that turns out to be hot can be promoted to a column later without +re-deriving the data. + +### Owned by `@hypaware/claude` {#owned-by-claude} + +**The dataset is contributed by the `@hypaware/claude` manifest and registered +at activation**, its first dataset. The payload shapes are Claude Code's, so +the plugin that already interprets them owns the table. Registration sets the +source signal, so the rows forward centrally by the same rules message rows +follow. + +### Consumers come later {#consumers-later} + +**Shipping the dataset is the whole of this decision.** No report, graph +projection, or status surface reads it yet. Recording the signal is cheap and +irreversible in the other direction: data not captured today cannot be +back-queried tomorrow. + +## Consequences + +- `hyp query sql` answers questions like "which tool calls did I reject this + week" without touching `ai_gateway_messages`. +- Two datasets now describe one session, joined on `session_id`, which is the + join a report or graph consumer will use. +- The `@hypaware/claude` plugin gains a dataset registration path it did not + have, so its activation now has a cache-writing responsibility as well as an + attach one. +- An upstream event we do not model still lands: unknown names keep their + attributes in JSON rather than being discarded. diff --git a/llp/0256-session-ignore-reaches-the-listener.decision.md b/llp/0256-session-ignore-reaches-the-listener.decision.md new file mode 100644 index 00000000..00480c04 --- /dev/null +++ b/llp/0256-session-ignore-reaches-the-listener.decision.md @@ -0,0 +1,63 @@ +# LLP 0256: Session ignore reaches the claude listener over the same control route + +**Type:** Decision +**Status:** Accepted +**Systems:** Privacy, Plugins, Sources, CLI +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0049, LLP 0066, LLP 0262 (the RFC this decision realizes, at +its open question 4, accepted 2026-08-17), LLP 0253, LLP 0257 +**Tracker:** hyparam/hypaware#798 (the implementation spec that settled this) + +> The claude telemetry listener hosts the same +> `/_hypaware/ignore/session` control route the gateway proxy hosts, over its +> own in-memory ignored-session set. `hyp session ignore` and +> `hyp session unignore` post to every listener that offers the route, and a +> partial success is reported, not swallowed. + +## Context + +LLP 0066's opt-out is an in-memory set living in the process that records the +exchange. With a second recorder in the picture, "don't record this +conversation" has to reach both, and only the recorders can answer whether it +did. + +## Decision + +### The listener hosts the route {#control-route-on-listener} + +**The claude listener serves the same route shape, verbs, and response body as +the gateway control route**: `GET`, `POST`, and `DELETE` on +`/_hypaware/ignore/session`, idempotent mutations, and a +`{ session_id, ignored, total }` reply. One shape means one client, one skill, +and one set of tests. It is loopback-only for the same reason the listener is. + +### The CLI posts to both {#cli-posts-to-both} + +**`hyp session ignore` / `unignore` addresses every listener that offers the +route and reports each outcome.** A machine mid-migration has both recorders +live at once, so ignoring on one is not ignoring. A listener that is not +running is not a failure (it is recording nothing); a listener that is running +and refuses is. + +### Still in memory, still nothing on disk {#in-memory-only} + +**The set stays in memory and dies with the process, exactly as LLP 0066 +requires.** No new on-disk contract is introduced. The durable expressions of +the same intent remain `.hypignore` and the machine-local list. + +### An ignored session's bodies are deleted {#bodies-deleted} + +**Ignoring a session makes its spooled bodies a deletion target, not a skip +target** (LLP 0253 #delete-on-drop). Without that, the transport works and the +content stays. + +## Consequences + +- `/hypaware-ignore` keeps working unchanged from the user's side across the + attach-mode switch, which is the point. +- The ignore reaches only sessions whose ids the recorder can see; an exchange + already projected before the ignore arrives is still recorded, exactly as + under LLP 0066 today. +- A second host of the route means the route's tests move to a shared shape + rather than being duplicated per plugin. diff --git a/llp/0257-claude-telemetry-listener-source.spec.md b/llp/0257-claude-telemetry-listener-source.spec.md new file mode 100644 index 00000000..ae299aa9 --- /dev/null +++ b/llp/0257-claude-telemetry-listener-source.spec.md @@ -0,0 +1,141 @@ +# LLP 0257: The Claude telemetry listener source + +**Type:** Spec +**Status:** Accepted +**Systems:** Sources, Plugins, Privacy, Observability +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0012, LLP 0015, LLP 0016, LLP 0021, LLP 0030, LLP 0032, +LLP 0049, LLP 0066, LLP 0103, LLP 0262 (the RFC this spec realizes, accepted 2026-08-17), LLP 0252, LLP 0253, LLP 0254, LLP 0255, LLP 0256, LLP 0258 +**Tracker:** hyparam/hypaware#798 + +> The source that receives Claude Code's own telemetry: an OTLP http/json +> listener plus a body-file reader, registered by `@hypaware/claude`, producing +> `ai_gateway_messages` rows and `claude_telemetry_events` rows. This is the +> requirements document the listener implementation and its tests answer to. +> The decisions it composes are LLP 0252 through LLP 0256 plus LLP 0258; the +> rationale lives there and in LLP 0262, not here. + +## Summary + +A **listener source** in the sense of LLP 0012: it owns a daemon lifecycle, +implements `start`, returns a `StartedSource`, and writes rows into the +intrinsic cache. It never sees sinks. What is new is that one source has two +inputs (an HTTP endpoint and a directory) and two outputs (two datasets). + +## Ownership and registration {#registration} + +- **S1** The source is contributed by `@hypaware/claude` through the kernel + source registry, with its own name and its own config section. The plugin + also contributes and registers the `claude_telemetry_events` dataset + (LLP 0255 #owned-by-claude). +- **S2** The OTLP http/json server machinery (routing, content-type and + encoding handling, `partialSuccess` envelopes) is shared with + `@hypaware/otel` rather than copied. Payload interpretation is + Claude-owned. +- **S3** The listener binds loopback-only, on its own port, separate from the + `@hypaware/otel` receiver and from the gateway. The port is config with a + default; `0` requests a dynamic port and the bound port is what attach + writes into the settings `env` block (LLP 0258 #env-keys). +- **S4** The self-telemetry loop guard of LLP 0021 applies: the daemon's own + exports must never be ingested by this listener. + +## Endpoint contract {#endpoint} + +- **S5** Accepts OTLP over HTTP with `Content-Type: application/json` on the + logs and metrics paths. Protobuf is out of scope, as it is for the existing + OTLP receiver. +- **S6** Serves the session-ignore control route on the same listener, + identical in shape to the gateway's (LLP 0256 #control-route-on-listener). +- **S7** Rejects non-loopback peers, and answers anything else with a + well-formed OTLP error rather than a crash: an exporter that cannot be + fixed from our side must not be able to stop the daemon. + +## Ingest {#ingest} + +- **S8** Events are the primary producer and are projected once each + (LLP 0252 #events-first). Row identity comes from `message.uuid`. +- **S9** Body files named by `api_request_body.body_ref` are read only for + `system_text`, the `tools` list, message ordering, and untruncated tool + arguments, then deleted (LLP 0252 #bodies-for-gaps, + LLP 0252 #project-then-delete). +- **S10** The usage-policy check runs inline before any row is written, using + the cwd recorded by the SessionStart hook (LLP 0254 #policy-inline). A + session with no hook record is undetermined, not clean. +- **S11** A dropped session's spooled bodies are deleted (LLP 0253 + #delete-on-drop). +- **S12** The spool is created `0700` under the HypAware home, capped, and + evicted oldest-first (LLP 0253 #spool-location, LLP 0253 #byte-cap). + +## Outputs {#outputs} + +- **S13** `ai_gateway_messages` rows carry the same projected-exchange values + the live proxy and backfill producers yield today, with unchanged dedupe, + partitioning, and repo identity columns (LLP 0252 #projection-unchanged). +- **S14** `claude_telemetry_events` rows are one per event, hot fields typed, + the remainder in an `attributes` JSON column, with the source signal set for + central forwarding (LLP 0255 #row-shape). +- **S15** `parent_uuid`, `logical_parent_uuid`, `user_type`, and + `permission_mode` read null on this path by design. + +## Status and capture health {#status-and-health} + +- **S16** `status()` reports `state`, `rowsWritten`, and details carrying the + bound address, the last event seen, and the spool's current byte size and + eviction count. +- **S17** `hyp status` renders a capture-health line comparing last event seen + against last transcript activity, and raises a diagnostics entry with a + severity when the gap exceeds a threshold. Status keeps answering from the + status file only. +- **S17b** `hyp status` also compares the endpoint the client's own attach + marker exports to against the port a live listener is bound to, and warns + when they disagree. S17's gap line detects the same failure only after a + threshold of transcript activity and cannot name its cause; both ports are + already on disk, so the direct comparison is exact and immediate. This is + LLP 0114 §fallback-is-visible applied to this listener: the fallback bind + that moves the port is precisely what leaves the client exporting to a port + some other process now holds. +- **S17c** The gap of S17 is measured from the newest of three moments: the + last event seen, the attach, and the *running* listener's own start. The + last of those is required because `last_event_at` is in-process state, so + every daemon restart republishes it as null however long capture has been + healthy; without it a long-attached machine reports an `error`-severity gap + after a routine `hyp daemon restart`, which is itself the first repair the + gap prints. The listener's start counts only while its daemon is alive - on + a dead one the growing gap is the finding, not an excuse. + +## Failure modes {#failure-modes} + +- **S18** Delivery is best effort. A down daemon loses events; content is + recovered by transcript backfill, and behavioral-event loss in that window + is accepted (LLP 0262 open question 1). +- **S19** A body that cannot be parsed or projected is deleted and counted, + not retried forever (LLP 0252 #project-then-delete). +- **S20** An unrecognized event name is recorded in + `claude_telemetry_events` with its attributes rather than discarded. +- **S21** Upstream shape drift is detected two ways: the capture-health line + in production, and a release-gate shape assertion against the installed + Claude Code. + +## Observability {#observability} + +- **S22** Per the repository's log-driven development rules, the listener + emits structured signals at the boundaries that can fail: listener start, + event batch received, body projected, body evicted, policy drop, dataset + write, and control-route mutation, each with `component`, `operation`, + `status`, and where applicable `error_kind`. +- **S23** No signal records credentials, raw prompt text, or hidden reasoning. + Payload identity is carried by hashes or short redacted excerpts. + +## Testing {#testing} + +- **S24** The primary seam is a hermetic smoke: POST OTLP/JSON at the + listener, drop body fixtures into the spool, drive the SessionStart hook, + then assert rows out of `hyp query sql`, body deletion, and the capture + spans. Tests never reach into projector internals. +- **S25** Privacy smokes cover the ignored-session case both ways + (`.hypignore` and the control route): only clean rows land, the drop signal + fires, and the ignored session's bodies are gone from the spool. +- **S26** Deterministic parts are unit tested in the root suite: spool cap + eviction order, event-plus-body projection identity, capture-health + rendering and its threshold. diff --git a/llp/0258-attach-injects-telemetry-via-settings-env.decision.md b/llp/0258-attach-injects-telemetry-via-settings-env.decision.md new file mode 100644 index 00000000..60b7320f --- /dev/null +++ b/llp/0258-attach-injects-telemetry-via-settings-env.decision.md @@ -0,0 +1,90 @@ +# LLP 0258: Attach turns on Claude Code telemetry by writing the settings `env` block + +**Type:** Decision +**Status:** Accepted +**Systems:** Config, Plugins +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0044, LLP 0045, LLP 0163, LLP 0232, LLP 0237, LLP 0239, +LLP 0262 (the RFC this decision realizes, accepted 2026-08-17) + +> `hyp attach claude` gains a third mode, `otel`, that merges a fixed set of +> telemetry keys into the `env` block of `~/.claude/settings.json` and writes +> nothing else. No PATH shim, no process wrapper, no keychain, no launchd +> environment. Below the Claude Code version floor attach refuses the switch +> instead of degrading. + +## Context + +See LLP 0262 for why the `claude` client leaves proxy attach at all. This +decision settles only the injection mechanism: which surface carries the +telemetry configuration, and what attach is allowed to touch to put it there. + +## Decision + +### The keys, and only these keys {#env-keys} + +**Attach merges this key set into `env` and manages exactly it.** + +``` +CLAUDE_CODE_ENABLE_TELEMETRY=1 +OTEL_LOGS_EXPORTER=otlp +OTEL_METRICS_EXPORTER=otlp +OTEL_EXPORTER_OTLP_PROTOCOL=http/json +OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1: +OTEL_LOG_USER_PROMPTS=1 +OTEL_LOG_ASSISTANT_RESPONSES=1 +OTEL_LOG_TOOL_DETAILS=1 +OTEL_LOG_RAW_API_BODIES=file:/spool/claude-bodies +``` + +The list is the decision, so it is written out here rather than described. +`ANTHROPIC_BASE_URL`, `HTTPS_PROXY`, and `NODE_EXTRA_CA_CERTS` are not written, +which is what keeps the endpoint first-party and Remote Control working without +the override keys LLP 0045 needed and LLP 0232 removed. + +### Settings `env` is the injection surface {#settings-env} + +**The `env` block of `settings.json` is the only place attach puts these +values.** Claude Code applies it at startup for every launch path (terminal, +desktop-spawned, SDK, background service), so one file write reaches sessions a +shell profile or a PATH shim never sees, and a running terminal app does not +have to be quit and reopened (the LLP 0239 duty this retires for the `claude` +client). Fleets deliver the same block through managed settings. + +### Nothing else is touched {#nothing-else} + +**Attach writes no keychain trust, no `launchctl setenv`, and no LaunchAgent.** +Those exist to make TLS interception work; with no interception there is +nothing for them to do, and each one is a standing obligation on the user's +machine that this mode declines to create. + +### The marker keeps being the whole undo record {#marker-and-spool} + +**The `otel` mode reuses the `_hypaware` marker unchanged in shape**: managed +env keys, `prev_env` per-key backup, atomic mtime-gated write, JSONC refusal, +malformed-block backup, and the mode-switch key release of LLP 0232 +#mode-migration. The marker additionally records the spool directory, because +detach and `hyp purge` have to sweep a path they did not compute (LLP 0253). +The core disk-driven detach replays the marker without knowing any key by name, +so detach needs no new adapter code. + +### Below the version floor attach refuses {#version-floor} + +**Claude Code older than 2.1.193 (2.1.214 for `tool_source` detail) makes attach +refuse the mode switch**: any existing attach is left exactly as it is, and the +run prints an upgrade hint (`claude update`). There is no proxy fallback for the +`claude` client. One attach mode per client keeps the test matrix single, and a +silent downgrade to a mode that captures less is the failure this refusal +exists to prevent. + +## Consequences + +- Attaching Claude Code raises no macOS security dialog and needs no `sudo`. +- A machine migrating from proxy mode still has a trusted CA in its login + keychain until `detach --purge` runs; migration offers that step (LLP 0262), + it does not perform it silently. +- `hyp status` reports the third mode, so a machine can be seen to be on + `otel`, `proxy`, or `base_url` attach. +- Codex is untouched and stays on base-URL attach, so three mechanisms now + coexist behind one marker format. diff --git a/llp/0262-otel-attach-replaces-proxy.rfc.md b/llp/0262-otel-attach-replaces-proxy.rfc.md new file mode 100644 index 00000000..caa9840b --- /dev/null +++ b/llp/0262-otel-attach-replaces-proxy.rfc.md @@ -0,0 +1,271 @@ +# LLP 0262: OTEL telemetry replaces proxy attach for Claude Code + +**Type:** RFC +**Status:** Accepted +**Systems:** Gateway, Sources, Config, Plugins, Privacy, Observability +**Author:** Phil / Claude +**Date:** 2026-08-17 +**Related:** LLP 0231, LLP 0242, LLP 0243, LLP 0244, LLP 0027, LLP 0030, LLP 0032, LLP 0049, LLP 0085, LLP 0103, LLP 0012, LLP 0015; formerly numbered 0245, ceded to PR #815 (2026-08-17) +**Tracker:** hyparam/hypaware#798 (implementation spec) +**Spawns:** LLP 0252, LLP 0253, LLP 0254, LLP 0255, LLP 0256, +LLP 0258 (decisions), LLP 0257 (spec). All accepted with this RFC, 2026-08-17. + +> Claude Code now ships a sanctioned, documented export path for everything the +> proxy intercepts: OTEL telemetry events plus raw API body files +> (`OTEL_LOG_RAW_API_BODIES=file:`). Attaching via that path needs no CA in +> the keychain, no `HTTPS_PROXY`, no launchd env, and no terminal restart: one +> `env` block written into `~/.claude/settings.json` reaches every session, +> however it is launched. This RFC proposes replacing proxy attach for the +> `claude` client with an OTEL attach, while the gateway proxy remains for every +> other client. + +## Context + +Proxy-mode capture (LLP 0231, default since LLP 0242 to 0244) exists because +repointing `ANTHROPIC_BASE_URL` broke Remote Control. It works, but it carries +standing costs that are intrinsic to TLS interception, not bugs to fix: + +- A machine-local CA trusted in the login keychain (one dialog at attach, a + `detach --purge` obligation at the end of life). +- `HTTPS_PROXY` plus `NODE_USE_SYSTEM_CA` in the launchd env, which never + reaches new windows of an already-running terminal app; a full quit and + reopen is required and is undetectable from our side (LLP 0231 run G + finding). +- The daemon sits on the wire for every request: if it is down or wedged, + Claude Code's traffic is affected, not just our capture. +- The capture depends on Claude Code not changing its proxy and CA handling, + which is behavior we consume but Anthropic does not promise us. + +Meanwhile Claude Code's telemetry system (docs: `monitoring-usage.md`) grew the +missing piece: raw request and response bodies, exportable untruncated to local +files, alongside an event stream that carries identity, cost, and behavioral +data the wire never shows. + +Sequencing (settled 2026-08-17): PR #794 shipped the proxy default as a +stopgap. If this RFC succeeds, OTEL attach is the successor migration for the +`claude` client in a later release; LLP 0244's migration machinery gets an +`Extended-by:` forward ref on acceptance, not a revert. + +### What was validated before this was proposed + +Spike run 2026-08-17 on Claude Code 2.1.233, one real session captured +simultaneously by the proxy (control) and by a scratch OTLP http/json listener +plus body-file spool (candidate): + +- Events observed: `user_prompt`, `assistant_response`, `api_request`, + `api_request_body`, `api_response_body`, `tool_decision`, `tool_result`, + `permission_mode_changed`, `mcp_server_connection`, `plugin_loaded`, and + (beyond the docs) `hook_registered`, `hook_execution_start`, + `hook_execution_complete`. +- Every event carried `session.id`, `prompt.id`, `user.email`, + `organization.id`, `user.account_uuid`, `terminal.type`, `app.version`, + `app.entrypoint`; content events carried full prompt text, full response + text, and full `tool_input` JSON (with `OTEL_LOG_USER_PROMPTS`, + `OTEL_LOG_ASSISTANT_RESPONSES`, `OTEL_LOG_TOOL_DETAILS` set). +- Body files held the complete request JSON: 4 system blocks, 12 tool + definitions, full message history, and `metadata.user_id` embedding + session id, account uuid, and device id. About 145 KB per request for a + trivial session (system prompt and tool definitions dominate). +- Thinking parity: the response body carries thinking blocks as + `"thinking":""` with the signature kept. The proxy control capture + of the same session stored its 2 reasoning parts with empty text. The wire + no longer carries thinking either (checked across August: 7201 of 7201 + captured Claude reasoning parts are empty). Neither path loses anything the + other has. +- `workspace.host_paths` did NOT appear on any event, so cwd and git identity + do not come from event attributes on a plain local session. + +## Requirements + +- **R1 Field parity.** Every `ai_gateway_messages` column populated by the + proxy path today is populated by the OTEL path, from events, body files, or + the retained SessionStart hook. See the parity table below. +- **R2 More when useful.** Net-new data with clear report or graph value is + captured, not discarded: tool accept/reject decisions with their source, + permission mode changes, active time, per-request USD cost, lines of code, + commit and PR counts, user email and org id, terminal type, hook execution, + MCP server health, refusals. +- **R3 Storage must not explode.** Steady-state Iceberg growth stays at or + below the proxy path's (the local all-history table is 182 MB today; August + is 45,200 rows). The transient body-file spool is bounded by a hard cap with + oldest-first eviction; eviction degrades to transcript backfill, never to + unbounded disk. +- **R4 Privacy seams hold.** `.hypignore` and the machine-local list (LLP + 0049, 0103), local-only withholding, `hyp purge`, and per-session ignore all + keep working. The policy check runs inline at ingest with cwd in hand; the + fail-open window LLP 0085 patches must not reappear. Settled 2026-08-17: + transient spool presence of a to-be-dropped session's bodies is acceptable + (the same content already sits in `~/.claude/projects` transcripts), with + three duties: the spool lives under hyp-home with owner-only permissions, + ingest DELETES (never merely skips) the bodies of ignored or policy-dropped + sessions, and `hyp purge` and detach both sweep the spool. +- **R5 Attach is one reversible write.** `hyp attach claude` merges keys into + the `env` block of `~/.claude/settings.json`; detach removes exactly those + keys. No PATH shim, no corporate launcher, no keychain, no launchd env, no + terminal restart. +- **R6 Remote Control untouched.** The base URL is never repointed and no + proxy is set, so the first-party predicate holds trivially. The override + keys LLP 0231 documents are not needed. + +## Proposal + +### Injection + +Attach writes into `~/.claude/settings.json`: + +``` +"env": { + "CLAUDE_CODE_ENABLE_TELEMETRY": "1", + "OTEL_LOGS_EXPORTER": "otlp", + "OTEL_METRICS_EXPORTER": "otlp", + "OTEL_EXPORTER_OTLP_PROTOCOL": "http/json", + "OTEL_EXPORTER_OTLP_ENDPOINT": "http://127.0.0.1:", + "OTEL_LOG_USER_PROMPTS": "1", + "OTEL_LOG_ASSISTANT_RESPONSES": "1", + "OTEL_LOG_TOOL_DETAILS": "1", + "OTEL_LOG_RAW_API_BODIES": "file:/spool/claude-bodies", + ... +} +``` + +Settings `env` overrides the shell environment at startup and reaches every +session regardless of launch path (terminal, desktop-spawned, SDK, background +service), which is strictly broader than the PATH-shim idea and does not need +the corporate launcher (`CLAUDE_CODE_PROCESS_WRAPPER`); that mechanism wraps +processes, which we do not need for env injection. Fleets can deliver the same +block via managed settings, with the documented approval dialog. + +### Capture + +A new listener source (home: the `@hypaware/claude` plugin, since the payload +shapes are Claude Code's, with the OTLP endpoint itself possibly shared +kernel machinery) receives the event stream and tails the body spool: + +- **Events first.** The event stream is naturally incremental: each piece of + content is emitted exactly once (`user_prompt` once, `assistant_response` + once, `tool_result` once), so it arrives pre-deduplicated. +- **Bodies for what events lack.** `system_text`, the `tools` list, message + ordering, and untruncated tool args come from the body files referenced by + `api_request_body.body_ref`. A body file is projected and then deleted. +- **Projection unchanged.** The source yields the same + `ai_gateway.projected_exchange` values backfill providers yield today; the + dataset, `part_id` dedupe, partitioning (LLP 0030), and repo identity + columns (LLP 0032) are untouched. OTEL is a third producer, not a new table. +- **Behavioral events** (`tool_decision`, `permission_mode_changed`, hook + execution, MCP health, and the metrics) land in their own dataset(s), not + crammed into `ai_gateway_messages`. Shape is an open question below. + +### Enrichment shrinks + +Native `message.uuid` on events kills the identity race the flush-time +settlement enricher exists for (LLP 0027). The `.hypignore` check moves inline +to ingest with cwd known (R4), retiring the late-drop machinery of LLP 0085 on +this path. What survives: transcript backfill as the recovery path, and the +SessionStart hook as the source of cwd and git identity (`git_remote`, +`head_sha`, `git_branch`), since events do not carry them (spike finding) and +deriving them from body system text is parseable but fragile. + +### Storage budget (R3) + +- Iceberg: identical mechanics to today. Request bodies repeat the full + history each turn, exactly as the proxied wire does, and the same `part_id` + dedupe stores each part once. Events reduce pressure further because they + never repeat content at all. +- Spool: bodies are large transiently (about 145 KB per request measured, so + a heavy day can pass gigabytes through the directory) but are deleted on + projection. The daemon enforces a byte cap with oldest-first eviction for + the daemon-down window; evicted bodies are recovered later from transcript + backfill. The cap is a config value with a sane default (proposal: 512 MB). + +### Migration + +`hyp attach claude` on a proxy-attached machine removes the proxy env keys, +unwinds launchd env, offers `detach --purge` for the CA trust, and writes the +OTEL env block. Sessions started before the flip keep proxying until restart; +capture overlap is harmless because both producers dedupe into the same rows. +The gateway proxy remains fully supported for codex, claude-desktop, openclaw, +hermes, and raw SDK traffic: this RFC narrows the proxy's client list, it does +not retire the gateway. + +## Field parity (R1) + +| Column(s) | OTEL source | +| --- | --- | +| `session_id`, `message_id`, `provider_uuid`, `request_id`, `prompt_id` | event attributes (`session.id`, `message.uuid`, `request_id`, `prompt.id`) | +| `model`, `role`, `content_text`, `part_*` | events + body messages | +| `system_text`, `tools` | body files | +| `tool_name`, `tool_call_id`, `tool_args`, `tool_result_for` | `tool_result` events + body blocks | +| usage tokens, `attributes.usage` | `api_request` events (plus body `usage`) | +| `cwd`, `git_branch`, `git_remote`, `head_sha`, `repo_root` | SessionStart hook (unchanged) | +| `client_version`, `entrypoint` | `app.version`, `app.entrypoint` (today: enrichment) | +| `user_id` | `user.account_uuid` / body `metadata.user_id` | +| `is_sidechain`, `agent_id` | `query_source`, `agent.name` (today: transcript inference) | +| `parent_uuid`, `logical_parent_uuid`, `user_type`, `permission_mode` | transcript join only, if retained (open question 3) | +| `thinking_signature` | body thinking blocks (text redacted both paths) | +| `raw_frame` | body excerpt at projection, same policy as today | + +Net-new (R2): everything listed in R2, none of it visible on the wire. + +## Alternatives considered + +- **Keep proxy attach, add OTEL alongside.** Captures the same content twice + and keeps every proxy cost. The behavioral event stream is worth ingesting + regardless, but content capture does not need two producers. +- **PATH shim plus corporate launcher.** The docs' own pattern for process + wrapping, but strictly more moving parts than the settings `env` block for + env injection, with worse coverage (shim misses GUI launches; launcher + misses plain terminal sessions by design). +- **Events only, no body files.** Cleanest storage story, but loses + `system_text`, the `tools` list, and untruncated tool args + (`tool_input` in events clips values at 512 chars, about 4 KB total). + Revisit if `CLAUDE_CODE_OTEL_CONTENT_MAX_LENGTH` plus future event + coverage closes those gaps. + +## Open questions + +1. **Delivery guarantees: resolved 2026-08-17.** Best-effort delivery is + accepted. Content loss has a recovery path (spool survives a down daemon; + transcript backfill covers the rest, as it already does for pre-attach + history); behavioral-event loss during daemon downtime is tolerated. + Duty: `hyp status` grows a capture-health line (last event seen vs last + transcript activity) so silent gaps are visible instead of discovered at + report time. +2. **Behavioral dataset shape: resolved 2026-08-17.** A new + `claude_telemetry_events` dataset owned by `@hypaware/claude`: one row per + event, typed columns for the hot fields (event name, session_id, + tool_name, decision, source, cost), attributes JSON for the rest. Not + routed through `@hypaware/otel`'s generic logs/metrics datasets, and not + widened into `ai_gateway_messages`. +3. **Parent chains: resolved 2026-08-17.** Code survey found no consumer of + `parent_uuid` / `logical_parent_uuid` outside their producers, and no graph + plugin reads `is_sidechain`. The columns stay in the schema and read null + on the OTEL path; `query_source` and `agent.name` are the attribution + source. No transcript join is kept for live capture. +4. **Session ignore: resolved 2026-08-17.** The claude listener hosts the + same session-ignore control route the gateway proxy hosts, and + `hyp session ignore` / `unignore` posts to both. Reuses the existing + in-memory mechanism; no new on-disk contract. (Settled in the + implementation spec, hyparam/hypaware#798.) +5. **Flag stability (position).** `OTEL_LOG_RAW_API_BODIES` and friends are + documented but young. Detection is two-layered: the `hyp status` + capture-health line (open question 1's duty) catches silent field drift in + production, and a hermetic smoke asserts the event and body shapes against + the installed Claude Code on every release. +6. **Version floor: resolved 2026-08-17.** Below the floor (>= 2.1.193 for + the event set, `tool_source` detail at >= 2.1.214), `hyp attach claude` + REFUSES to switch modes: it leaves any existing attach untouched and + prints an upgrade hint (`claude update`). No proxy fallback for the + `claude` client: one attach mode per client keeps the test matrix single, + and Claude Code self-updates aggressively enough that stale clients are + transient. + +## On acceptance + +This RFC stays the deliberation record and spawns six narrow decisions, one per +settled choice: injection mechanism (LLP 0258), events-plus-bodies split +(LLP 0252), spool cap policy (LLP 0253), settlement retirement scope +(LLP 0254), behavioral dataset shape (LLP 0255), and session-ignore transport +(LLP 0256, the choice open question 4 above resolves). It also spawns a spec +for the listener source (LLP 0257). The proxy-attach docs it displaces for the +`claude` client (parts of LLP 0231 to 0244) get `Extended-by:` / +`Superseded-by:` forward refs at that point, not before. diff --git a/llp/0263-hook-enforces-the-body-spool-cap.decision.md b/llp/0263-hook-enforces-the-body-spool-cap.decision.md new file mode 100644 index 00000000..e7710901 --- /dev/null +++ b/llp/0263-hook-enforces-the-body-spool-cap.decision.md @@ -0,0 +1,103 @@ +# LLP 0263: The client hook is the body spool's second cap enforcer + +**Type:** Decision +**Status:** Accepted +**Systems:** Privacy, Sources, Daemon, Config +**Author:** Phil / Claude +**Date:** 2026-08-18 +**Related:** LLP 0085, LLP 0253 (the decision this extends), LLP 0258, LLP 0262 + +> The raw-body spool's byte cap is enforced by the `hyp claude-hook +> session-context` hook as well as by the daemon. LLP 0253 named the daemon as +> its enforcer and named the daemon-down window as the reason the cap exists, +> which are the same sentence contradicting itself. The hook already runs +> out-of-process at the cadence bodies are written, so it closes the window at +> no new cost and under the operator's existing cap. + +## Context + +LLP 0253 #byte-cap settles that the spool is bounded, and says why: "The cap is +enforced by the daemon, not by hoping the reader keeps up: the window this +exists for is precisely the one where the reader is not running." + +As built, every enforcement of that cap lives inside the listener source: a +one-shot sweep when the source starts and a 60-second timer cleared on stop. +The only other sweeps are `hyp purge` and `hyp detach`, both user-initiated. +Nothing else touches the directory. + +So the window LLP 0253 names is exactly the window nothing swept. Claude Code +keeps writing bodies whether or not the daemon is reading (LLP 0253, Context), +at roughly 145 KB per request, and the daemon is legitimately absent in more +than failure states: + +- a crashed or stopped daemon service, +- a machine where the daemon was installed but never started, +- an uninstall that never ran `hyp detach`, leaving the settings block in + place with no reader that will ever return, +- attach before the first daemon start, which + `resolveAttachTelemetryPort`'s third rung deliberately supports. + +This is a privacy defect, not only a disk one. The attach turns on +`OTEL_LOG_USER_PROMPTS`, `OTEL_LOG_ASSISTANT_RESPONSES`, `OTEL_LOG_TOOL_DETAILS` +and `OTEL_LOG_RAW_API_BODIES` (LLP 0258 #env-keys). LLP 0262 accepted that +content sitting in our own directory only because it is transient, and LLP 0253 +promised "Nothing in the spool outlives the user's decision to remove it". +With no daemon it is neither transient nor bounded, in a directory a +non-excluding backup tool will copy. + +## Decision + +### The client hook enforces the cap too {#hook-enforces-the-cap} + +**`hyp claude-hook session-context` enforces the spool's byte cap on every +invocation, in addition to the daemon.** The hook is chosen over the other +daemon-less touchpoints because it is the only one whose cadence is tied to the +writing itself: attach installs it on `SessionStart`, `CwdChanged`, +`UserPromptSubmit`, and `PostToolUse` for Bash (LLP 0085), so bodies cannot +accumulate between enforcements. Enforcing at `hyp attach` or `hyp status` +instead was rejected: both are typed rarely or once, so a machine that attaches +and is never inspected again gets no bound at all, which is the case this +exists for. + +**A hook may delete spool files, and only spool files.** It calls the same +`enforceClaudeBodySpoolCap` over the same directory, at the same cap, in the +same oldest-first order. The hook never widens the deletion rule; it runs the +daemon's existing rule while the daemon cannot. Everything LLP 0253 +#eviction-degrades already says about an evicted body applies unchanged: the +content is recoverable from the transcript by the backfill path. + +**The cap the hook applies is the operator's.** It reads the same +`telemetry.spool_max_bytes` key out of the `@hypaware/claude` slice of the v2 +config, with the same validation and the same 512 MB default, so lowering the +cap on a small disk binds both enforcers. A malformed value falls back +silently rather than warning: the listener already warns on this key, and a +hook has no output surface that would not push text at Claude Code. + +### The sweep runs last and may always fail {#never-interrupts} + +**The sweep runs after the session-context records are written, and a failure +is swallowed.** Ordering is not incidental: LLP 0085 exists to shrink the +window in which the projector reads a cwd-less record, so nothing may be added +ahead of those appends. Nothing waits on the sweep, so it goes last, and it +runs even on the invocations that record nothing, because a malformed event or +a missing `--state-file` says nothing about whether the spool is filling. + +The cost is one `readdir` plus a `stat` per file, against a directory the +listener keeps near-empty whenever the daemon is up, and which does not exist +at all on a proxy-attached or unattached machine (the sweep returns on its +`ENOENT` arm without a single stat). That is well under the two git +subprocesses the same hook already spawns. + +## Consequences + +- LLP 0253's stated bound holds in the window it was written for, so LLP 0262's + acceptance of transient spool presence rests on something true. +- A machine whose daemon never runs again still converges to the cap, at the + cost of one directory listing per hook event. +- Deleting captured data is no longer daemon-only. The rule is unchanged and + the directory is the one `hyp purge` and detach already empty, but a reviewer + looking for "who may delete captured content" now has two answers, both + pointed at this anchor. +- The two enforcers can disagree about the cap only if the hook and the daemon + read different configs or different `HYP_HOME`s, which is already true of the + state file and the cache and is not made worse here. diff --git a/src/core/capture_spool.js b/src/core/capture_spool.js new file mode 100644 index 00000000..563d1c75 --- /dev/null +++ b/src/core/capture_spool.js @@ -0,0 +1,118 @@ +// @ts-check + +import fsp from 'node:fs/promises' +import path from 'node:path' + +import { errCode } from './util/json_util.js' + +/** + * @import { Dirent } from 'node:fs' + */ + +/** + * The capture-spool root under the HypAware home: `/spool`. + * + * A client that writes raw request/response bodies for us drops them into a + * directory beneath it, one per client (`spool/claude-bodies` today). Core + * knows the root; the client directories are the plugins'. That split is what + * lets `hyp purge` empty every spool without naming a plugin, and what bounds + * the directory a detach may sweep from a path it read out of a settings file. + * + * @ref LLP 0253#spool-location [implements]: a path under the HypAware home is + * what `hyp purge` and detach can find without being told + */ +const SPOOL_ROOT_DIRNAME = 'spool' + +/** + * @param {string} hypHome + * @returns {string} + */ +export function captureSpoolRoot(hypHome) { + return path.join(hypHome, SPOOL_ROOT_DIRNAME) +} + +/** + * Whether `dir` names a capture spool this install owns: an absolute path + * whose parent is exactly `/spool`. + * + * The test exists because detach learns the path from the attach marker, which + * lives in the user's own settings file and is therefore reachable by a hand + * edit. Without a containment rule, "sweep the directory the marker names" + * would be a recursive-delete primitive pointed at an arbitrary path. Depth + * one, not "somewhere under the root", so a marker cannot walk the sweep down + * into a nested tree either; `path.resolve` normalizes any `..` away first, so + * an escaping spelling lands outside the root and fails the test. + * + * @param {unknown} dir + * @param {string} hypHome + * @returns {boolean} + */ +export function isCaptureSpoolDir(dir, hypHome) { + if (typeof dir !== 'string' || dir.length === 0 || !path.isAbsolute(dir)) return false + const resolved = path.resolve(dir) + return path.dirname(resolved) === path.resolve(captureSpoolRoot(hypHome)) +} + +/** + * Empty a capture spool: remove every file under `dir`, keeping the + * directories themselves. + * + * The directory survives because the client that writes into it was told its + * path at attach and is not asked again; only the contents are the user's data. + * The walk never throws - a spool sweep runs at the end of a destructive verb + * whose real work has already landed, so an unreadable subdirectory is a + * `failed` count the caller reports, not a reason to fail a purge or a detach + * that already succeeded. + * + * @ref LLP 0253#purge-and-detach-sweep [implements]: purge and detach both + * remove the spool directory's contents + * @param {string} dir + * @param {{ fs?: typeof fsp }} [opts] + * @returns {Promise<{ filesRemoved: number, bytesRemoved: number, failed: number }>} + */ +export async function sweepCaptureSpool(dir, opts = {}) { + const fs = opts.fs ?? fsp + let filesRemoved = 0 + let bytesRemoved = 0 + let failed = 0 + + /** @type {string[]} */ + const pending = [dir] + while (pending.length > 0) { + const current = /** @type {string} */ (pending.pop()) + /** @type {Dirent[]} */ + let entries + try { + entries = await fs.readdir(current, { withFileTypes: true }) + } catch (err) { + // An absent spool is the normal case on a machine that never attached a + // body-writing client; anything else is a directory we could not empty. + if (errCode(err) !== 'ENOENT') failed += 1 + continue + } + for (const entry of entries) { + const full = path.join(current, entry.name) + if (entry.isDirectory()) { + pending.push(full) + continue + } + // `lstat`, so a symlink is measured and removed as the link it is rather + // than followed out of the spool. + let size = 0 + try { + size = (await fs.lstat(full)).size + } catch (err) { + if (errCode(err) === 'ENOENT') continue + } + try { + await fs.rm(full, { force: true }) + filesRemoved += 1 + bytesRemoved += size + } catch { + failed += 1 + } + } + } + + return { filesRemoved, bytesRemoved, failed } +} diff --git a/src/core/commands/purge.js b/src/core/commands/purge.js index fca1bade..0527ba4c 100644 --- a/src/core/commands/purge.js +++ b/src/core/commands/purge.js @@ -10,6 +10,7 @@ import { isTty } from '../cli/stdio.js' import { Attr, getLogger, withSpan } from '../observability/index.js' import { readObservabilityEnv } from '../observability/env.js' import { purgeCache } from '../cache/purge.js' +import { captureSpoolRoot, sweepCaptureSpool } from '../capture_spool.js' import { createUsagePolicyResolver, localOnlyListPath } from '../usage-policy/index.js' /** @@ -34,6 +35,10 @@ import { createUsagePolicyResolver, localOnlyListPath } from '../usage-policy/in * purge-then-re-record is idempotent server-side and never resurrects rows via * a stale watermark. * + * "Cache-only" describes where it reaches, not that rows are the only thing it + * removes: it also empties the raw-body capture spool, which is a transit area + * holding bodies no row has been made from yet (LLP 0253). + * * @ref LLP 0104 [implements]: the `hyp purge` verb (targeted, cache-only, confirmed), with non-destructive marking left intact * @param {string[]} argv * @param {CommandRunContext} ctx @@ -46,7 +51,7 @@ export async function runPurge(argv, ctx) { return 2 } - const stateDir = readObservabilityEnv(ctx.env).stateDir + const { hypHome, stateDir } = readObservabilityEnv(ctx.env) const resolver = createUsagePolicyResolver({ localOnlyListPath: localOnlyListPath(stateDir) }) const target = buildTarget(parsed, ctx, resolver) @@ -91,12 +96,27 @@ export async function runPurge(argv, ctx) { return 1 } + // The capture spool, emptied whatever the target was. The files in it are + // raw request and response bodies that have not been projected yet, so + // leaving them would let the next batch write rows the user just deleted - + // and a targeted purge cannot tell which of them belong to its target, + // because a spooled body carries no cwd. They are transient by design and + // recoverable from the client's own transcript, so emptying them costs + // detail at worst. + // @ref LLP 0253#purge-and-detach-sweep [implements]: `hyp purge` removes the + // spool directory's contents + const swept = await sweepCaptureSpool(captureSpoolRoot(hypHome)) + getLogger('cache').info('purge.result', { [Attr.COMPONENT]: 'cmd-purge', [Attr.OPERATION]: 'purge.result', target_kind: target.kind, rows_deleted: summary.rowsDeleted, partitions_affected: summary.partitionsAffected, + // Counts and bytes only: a spooled body's filename is the client's, and + // its content is a raw prompt. + spool_files_removed: swept.filesRemoved, + spool_bytes_removed: swept.bytesRemoved, // A count, never a path. The near-miss decision is otherwise visible only // on stderr, so a smoke could assert the user-visible result without any // internal signal that the spelling predicate actually ran the branch. @@ -122,12 +142,30 @@ export async function runPurge(argv, ctx) { resurrectable, retainedAliasRows: summary.retainedAliasRows, retainedAliasCwds: retainedAliases, + spoolFilesRemoved: swept.filesRemoved, }) + '\n') } else { ctx.stdout.write( `purged ${summary.rowsDeleted} row${summary.rowsDeleted === 1 ? '' : 's'} ` + `from ${summary.partitionsAffected} partition${summary.partitionsAffected === 1 ? '' : 's'}\n` ) + // Reported only when it did something: a machine with no body-writing + // client attached has an empty (or absent) spool on every purge, and a + // standing "swept 0 files" line would train the reader to skip the line + // that matters on the machine where it is not zero. + if (swept.filesRemoved > 0) { + ctx.stdout.write( + `also emptied the capture spool: ${swept.filesRemoved} ` + + `raw body file${swept.filesRemoved === 1 ? '' : 's'} deleted\n` + ) + } + } + + if (swept.failed > 0) { + ctx.stderr.write( + `note: ${swept.failed} item${swept.failed === 1 ? '' : 's'} in the capture spool ` + + `(${captureSpoolRoot(hypHome)}) could not be removed; delete the directory by hand\n` + ) } // The near-miss report (LLP 0104 #spellings). Purge reaches a row recorded diff --git a/src/core/commands/status.js b/src/core/commands/status.js index 6f061bf1..52f702b1 100644 --- a/src/core/commands/status.js +++ b/src/core/commands/status.js @@ -193,6 +193,10 @@ export function renderStatusJson({ report, clientNames, datasets, cacheRoot }) { ...(c.settingsPath ? { settings_path: c.settingsPath } : {}), ...(c.version ? { version: c.version } : {}), ...(c.port ? { port: c.port } : {}), + // The attach mode the marker records (`base_url` / `proxy` / `otel`), + // so a machine can be seen to be on the third mode without opening + // the settings file (LLP 0258's consequence). + ...(c.mode ? { mode: c.mode } : {}), ...(c.error ? { error: c.error } : {}), })), // Picked clients grouped by provenance (LLP 0132 #never-silent). Null on @@ -211,6 +215,23 @@ export function renderStatusJson({ report, clientNames, datasets, cacheRoot }) { last_seen: e.lastSeen, rows: e.rows, })), + // Capture health per otel-attached client (LLP 0257 S17). Always an + // array so a consumer can pin the key; empty means no configured client + // is otel-attached, which keeps the pre-otel payload shape unchanged. + // Timestamps follow the null-not-omitted contract: `last_event_at: + // null` is the actionable answer ("attached, nothing ever arrived"), + // not a missing field. + // @ref LLP 0257#status-and-health [implements]: --json carries the machine-readable comparison the text line renders + capture_health: report.captureHealth.map((c) => ({ + client: c.client, + source: c.source, + last_event_at: c.lastEventAt, + last_transcript_activity_at: c.lastTranscriptActivityAt, + attached_at: c.attachedAt, + listener_started_at: c.listenerStartedAt, + gap_seconds: Math.round(c.gapMs / 1000), + state: c.state, + })), datasets: datasets.map((d) => ({ name: d.name, plugin: d.plugin })), cache: { dir: cacheRoot, @@ -417,9 +438,26 @@ export function renderStatusText({ report, clientNames, datasets, cacheRoot, std state.push(c.configured ? 'configured' : 'not in config') // A client with no attach probe has no attach state to report: printing // `not attached` for it invites a `hyp attach` that is a documented - // no-op and can never change the line (#544). + // no-op and can never change the line (#544). Where there is an attach, + // the marker's mode rides the attached state (`attached (otel)`), so a + // machine that just migrated modes is visibly on the new one from the + // surface a human reads, not only under --json (LLP 0258's consequence, + // completed by the LLP 0262 migration). Markers that predate modes carry + // none and keep the bare word. The mode goes through `printable` because + // it is read back off the client's own settings file, which a hand edit + // can fill with anything: an unsanitized value here would let a settings + // file drive the operator's terminal, which is what LLP 0225 exists to + // stop. A value that sanitizes away entirely leaves the bare word. // @ref LLP 0229#status-derives-by-the-same-gate [implements]: the clients row says attach n/a, not "not attached", for a probe-less client - state.push(c.attachable === false ? 'attach n/a' : c.attached ? 'attached' : 'not attached') + // @ref LLP 0225#one-vocabulary [implements]: a label lifted off disk is stripped before it reaches the terminal + const mode = printable(c.mode) + state.push( + c.attachable === false + ? 'attach n/a' + : c.attached + ? (mode ? `attached (${mode})` : 'attached') + : 'not attached' + ) stdout.write(` - ${c.name} [${state.join(', ')}]${provenanceTag(report.layered, isCentralPlugin(report.layered, c.plugin))}\n`) if (c.error) stdout.write(` error: ${c.error}\n`) } @@ -459,6 +497,29 @@ export function renderStatusText({ report, clientNames, datasets, cacheRoot, std } } + // Capture health for otel-attached clients (LLP 0257 S17): the line that + // answers "is the telemetry path keeping up with what the client itself is + // doing?", which no other line can be read for - `recent clients` above + // only knows what WAS captured, and a silent capture gap is precisely rows + // that never arrived. Rendered only when a configured client is + // otel-attached, so every other install's text surface is unchanged; the + // `[capture gap]` tag points at the diagnostics block, which carries the + // repair. + // @ref LLP 0257#status-and-health [implements]: hyp status renders last event seen vs last transcript activity + if (report.captureHealth.length > 0) { + stdout.write(' capture health:\n') + for (const c of report.captureHealth) { + const events = c.lastEventAt !== null + ? `last event ${formatEntrypointAge(c.lastEventAt)}` + : 'no events yet' + const transcripts = c.lastTranscriptActivityAt !== null + ? `last transcript activity ${formatEntrypointAge(c.lastTranscriptActivityAt)}` + : 'no transcript activity' + const tag = c.state === 'gap' ? ' [capture gap]' : '' + stdout.write(` - ${c.client} ${events}, ${transcripts}${tag}\n`) + } + } + // Proxy mode's two invisible preconditions (LLP 0237, LLP 0239). Rendered // only where the question applies - macOS with a CA on disk - so an ordinary // install's text output is unchanged and a Linux host is never told about a diff --git a/src/core/config/client_detach_disk.js b/src/core/config/client_detach_disk.js index 314865ff..0e479037 100644 --- a/src/core/config/client_detach_disk.js +++ b/src/core/config/client_detach_disk.js @@ -5,9 +5,11 @@ import os from 'node:os' import path from 'node:path' import process from 'node:process' +import { captureSpoolRoot, isCaptureSpoolDir, sweepCaptureSpool } from '../capture_spool.js' import { resolveClientSettingsPath } from '../daemon/client_settings_path.js' import { removeLaunchdEnv } from '../daemon/launchd_env.js' import { Attr, getLogger } from '../observability/index.js' +import { readObservabilityEnv } from '../observability/env.js' import { ConcurrentEditError, atomicWriteFile } from '../util/fs_atomic.js' import { errCode, getAtDottedPath, isPlainObject, redactUrlUserinfo } from '../util/json_util.js' import { isOwnedProviderEntry } from './provider_entry_ownership.js' @@ -325,6 +327,12 @@ async function detachJsonMarker({ settingsPath, markerKey, fs, env, homeDir, pla // @ref LLP 0239#launchctl-setenv [implements]: detach reverses the launchd env await releaseProxyModeLaunchdEnv({ marker, homeDir, warnings, platform, runCommand }) + // And the body spool an `otel`-mode attach pointed the client at. Same + // ordering rule: the settings write has landed, so the client is no longer + // producing bodies, and a sweep that fails leaves a warning rather than an + // un-detached client. + await sweepMarkerSpool({ marker, env, fs, warnings }) + const warning = joinWarnings(warnings) /** @type {DetachFromDiskResult} */ @@ -668,6 +676,11 @@ async function detachLegacyJsonMarker({ settingsPath, markerKey, value, marker, // still runs; the CA stays, exactly as on the record-driven branch. await releaseProxyModeLaunchdEnv({ marker, homeDir, warnings, platform, runCommand }) + // `spool_dir` is a top-level marker field, so it survives a damaged undo + // record the same way `mode` does. Sweeping it here is what keeps the one + // branch that reverses by convention from leaving raw prompt bodies behind. + await sweepMarkerSpool({ marker, env, fs, warnings }) + const warning = joinWarnings(warnings) /** @type {DetachFromDiskResult} */ @@ -733,6 +746,67 @@ async function releaseProxyModeLaunchdEnv({ marker, homeDir, warnings, platform, } } +/** + * Empty the raw-body spool an `otel`-mode attach recorded on its marker. + * + * The path comes off the marker rather than being recomputed, because the + * config that produced it is gone by the time detach runs and a machine whose + * HypAware home moved would otherwise sweep the wrong directory (or none). + * That makes the path *settings-file input*, which a hand edit can reach, so it + * is honored only when it is a direct child of this install's + * `/spool`: without that gate, "empty the directory the marker names" + * would be a recursive delete pointed anywhere. A path that fails the gate is + * left alone and reported, never guessed at. + * + * Best effort and never fatal, like the launchd release above: the settings + * undo has already landed, and a spool we could not empty is a leftover the + * user can be told about, not a reason to fail a detach that succeeded. + * + * @ref LLP 0253#purge-and-detach-sweep [implements]: detach removes the spool + * directory's contents, using the path the marker recorded + * @ref LLP 0258#marker-and-spool [constrained-by]: the marker records the spool + * directory precisely so this undo does not have to compute it + * @param {{ + * marker: Record, + * env: NodeJS.ProcessEnv | undefined, + * fs: typeof fsp, + * warnings: string[], + * }} args + */ +async function sweepMarkerSpool({ marker, env, fs, warnings }) { + const recorded = marker.spool_dir + if (recorded === undefined) return + + const { hypHome } = readObservabilityEnv(env) + if (!isCaptureSpoolDir(recorded, hypHome)) { + warnings.push( + `the attach marker names a body spool outside ${captureSpoolRoot(hypHome)}; ` + + 'it was left in place, so delete it by hand if it holds captured bodies' + ) + return + } + + const dir = /** @type {string} */ (recorded) + const swept = await sweepCaptureSpool(dir, { fs }) + if (swept.failed > 0) { + warnings.push( + `${swept.failed} item${swept.failed === 1 ? '' : 's'} in the body spool could not be removed; ` + + `empty ${dir} by hand` + ) + } + if (swept.filesRemoved === 0 && swept.failed === 0) return + // Counts, never filenames: a spooled body's name is the client's and its + // content is a raw prompt. + getLogger('client-detach').info('client.detach.spool_swept', { + [Attr.COMPONENT]: 'client-detach', + [Attr.OPERATION]: 'client.detach.spool_sweep', + [Attr.STATUS]: swept.failed > 0 ? 'partial' : 'ok', + files_removed: swept.filesRemoved, + bytes_removed: swept.bytesRemoved, + failed: swept.failed, + }) +} + /** * Reverse the proxy-mode env keys from a marker whose undo record is damaged. * diff --git a/hypaware-core/plugins-workspace/ai-gateway/src/control.js b/src/core/control/session_ignore.js similarity index 63% rename from hypaware-core/plugins-workspace/ai-gateway/src/control.js rename to src/core/control/session_ignore.js index 3454659d..c2accf29 100644 --- a/hypaware-core/plugins-workspace/ai-gateway/src/control.js +++ b/src/core/control/session_ignore.js @@ -2,46 +2,82 @@ /** * @import { IncomingMessage, ServerResponse } from 'node:http' - * @import { PluginLogger } from '../../../../hypaware-plugin-kernel-types.js' + * @import { PluginLogger } from '../../../hypaware-plugin-kernel-types.js' */ /** * The single V1 control route. The reserved `/_hypaware/` prefix is a - * LOCAL control surface (see `isControlPath` in proxy.js); this is the one + * LOCAL control surface (see `isControlPath` below); this is the one * endpoint served under it today. + * + * The route is hosted by every recorder that keeps an in-memory + * ignored-session set: the gateway proxy and the claude telemetry + * listener. One shape (verbs, body, reply) means one client, one skill, + * and one set of tests, which is why the handler lives in core rather + * than in either plugin. + * @ref LLP 0256#control-route-on-listener [implements]: the second host serves + * the identical route, so the handler is shared machinery, not a copy */ -const IGNORE_SESSION_PATH = '/_hypaware/ignore/session' +export const SESSION_IGNORE_CONTROL_PATH = '/_hypaware/ignore/session' + +/** + * The name a recorder advertises in its status details (`control_routes`) + * to say "I host the session-ignore route at my bound listener". The CLI's + * `hyp session ignore` / `unignore` discovers additional recorders by this + * advertisement (`resolveLiveControlRouteEndpointsFromStatus`), so offering + * the route is stated by the recorder itself, never guessed from a source + * name - which is what keeps the client-agnostic verb free of any list of + * client plugins. + * @ref LLP 0256#cli-posts-to-both [implements] + */ +export const SESSION_IGNORE_ROUTE = 'ignore/session' /** * Max request-body size for a control request. The skill sends a tiny * `{"session_id":"..."}` object; anything larger is rejected with 413 - * rather than buffered, so a stray large body cannot grow gateway memory. + * rather than buffered, so a stray large body cannot grow the hosting + * process's memory. */ const MAX_BODY_BYTES = 64 * 1024 /** - * Build the `onControlRequest` callback the proxy invokes for any request - * under the reserved `/_hypaware/` prefix (proxy.js short-circuits these - * BEFORE upstream matching, so a control request is never proxied and never - * starts an exchange). + * Recognize the reserved `/_hypaware/` local control prefix. Uses the same + * segment-boundary discipline as the gateway's `pathMatchesPrefix`: + * `/_hypaware` itself and any `/_hypaware/...` sub-path match, but + * `/_hypawarefoo` does not, so a look-alike upstream path is never mistaken + * for a control request. + * + * @ref LLP 0066#control-path [implements] + * @param {string} pathname + */ +export function isControlPath(pathname) { + return pathname === '/_hypaware' || pathname.startsWith('/_hypaware/') +} + +/** + * Build the control-request callback a hosting server invokes for any + * request under the reserved `/_hypaware/` prefix (the gateway proxy and + * the shared OTLP server both short-circuit these BEFORE their own + * routing, so a control request is never proxied, never starts an + * exchange, and never reads as an OTLP export). * * One route (`GET` / `POST` / `DELETE /_hypaware/ignore/session`) over * the in-memory `ignoredSessions` set. The mutating verbs are idempotent by * `Set` semantics (re-POSTing an ignored id or DELETEing an unknown id is a * 200 no-op); `GET` mutates nothing and answers the membership question for * one id. All three return `{ session_id, ignored, total }`; the skill reads - * `.total`. The `session_id` is an opaque token: the gateway never + * `.total`. The `session_id` is an opaque token: the host never * interprets it, keeping the LLP 0050 provider-agnostic boundary exact. * * **`ignored: true` is set membership, and is not a verified drop.** The - * gateway holds tokens, not traffic: the drop happens in the client adapter, - * keyed on the `session_id` it stamps on the row (LLP 0066 R5), so this route - * cannot tell a live session id from a Codex thread id or a typo and answers - * `ignored: true` for all three. Making it able to would mean teaching a - * deliberately provider-agnostic route about client grain, which is the - * boundary above. So the contract is the narrow one and the CALLER owns - * resolving the right key before it posts; responses that read as more than - * that are what LLP 0066 R14 forbids. + * route holds tokens, not traffic: the drop happens where the recorder + * resolves a `session_id` for the rows it is about to write (LLP 0066 R5), + * so this route cannot tell a live session id from a Codex thread id or a + * typo and answers `ignored: true` for all three. Making it able to would + * mean teaching a deliberately provider-agnostic route about client grain, + * which is the boundary above. So the contract is the narrow one and the + * CALLER owns resolving the right key before it posts; responses that read + * as more than that are what LLP 0066 R14 forbids. * @ref LLP 0066#receipt-is-membership [constrained-by]: the route confirms the * write only, so callers must resolve the key rather than expect an echo to * prove the drop. @@ -54,12 +90,17 @@ const MAX_BODY_BYTES = 64 * 1024 * @param {{ * ignoredSessions: Set, * log?: PluginLogger, - * }} opts + * logEvent?: string, + * logFields?: Record, + * }} opts `logEvent` / `logFields` let each host stamp its own identity on + * the mutation log; the defaults keep the gateway's original signal shape. * @returns {(req: IncomingMessage, res: ServerResponse, url: URL) => void} */ export function createControlHandler(opts) { const ignoredSessions = opts.ignoredSessions const log = opts.log + const logEvent = opts.logEvent ?? 'aigw.control.ignore_session' + const logFields = opts.logFields ?? { component: 'ai-gateway' } /** * @param {IncomingMessage} req @@ -67,7 +108,7 @@ export function createControlHandler(opts) { * @param {URL} url */ return function onControlRequest(req, res, url) { - if (url.pathname !== IGNORE_SESSION_PATH) { + if (url.pathname !== SESSION_IGNORE_CONTROL_PATH) { req.resume() sendJson(res, 404, { error: 'unknown control path', path: url.pathname }) return @@ -78,7 +119,7 @@ export function createControlHandler(opts) { // @ref LLP 0066#readable [implements]: the set is a privacy control, so it // must be readable, not only writable. `GET` answers "is this session // being dropped right now?" without mutating anything, which is what makes - // the two fail-open transitions - a gateway restart (LLP 0066#ephemeral) + // the two fail-open transitions - a host restart (LLP 0066#ephemeral) // and a session id that changed under the client - detectable instead of // silent. The id rides the query string rather than a body because a // read has no body; `URLSearchParams` round-trips the token byte-exactly, @@ -129,8 +170,8 @@ export function createControlHandler(opts) { ignored = false } const total = ignoredSessions.size - log?.info?.('aigw.control.ignore_session', { - component: 'ai-gateway', + log?.info?.(logEvent, { + ...logFields, operation: 'ignore_session', method, session_id: sessionId, @@ -144,20 +185,21 @@ export function createControlHandler(opts) { /** * Pull the opaque `session_id` token out of a parsed control-request body. - * The gateway never interprets the value; it only requires a non-empty + * The host never interprets the value; it only requires a non-empty * string (missing / empty / non-string → the caller returns 400). * * The returned value is the RAW string verbatim, NOT trimmed. Trimming is * used only to validate non-emptiness; the token itself must stay - * byte-identical to what the caller posted, because the adapters key the + * byte-identical to what the caller posted, because the recorders key the * drop on the RAW resolved session id (Claude's `resolveClaudeSessionId`, - * Codex's metadata/header readers) and none of them trim. Trimming here - * would desync the stored token from the adapter's lookup key: a - * whitespace-padded `session_id` would be stored trimmed but looked up raw, - * so `ignoredSessions.has()` would miss and the exchange would be RECORDED - * despite the opt-out, the privacy-relevant failure direction. + * Codex's metadata/header readers, the telemetry events' `session.id`) and + * none of them trim. Trimming here would desync the stored token from the + * recorder's lookup key: a whitespace-padded `session_id` would be stored + * trimmed but looked up raw, so `ignoredSessions.has()` would miss and the + * exchange would be RECORDED despite the opt-out, the privacy-relevant + * failure direction. * @ref LLP 0066#requirements: R5: the match key MUST be the session_id the - * adapter resolves and stamps, verbatim. + * recorder resolves and stamps, verbatim. * * @param {unknown} body * @returns {string | undefined} diff --git a/src/core/daemon/client_settings_path.js b/src/core/daemon/client_settings_path.js index 8a434f9e..ced987ca 100644 --- a/src/core/daemon/client_settings_path.js +++ b/src/core/daemon/client_settings_path.js @@ -65,19 +65,26 @@ export class ClientSettingsPathError extends Error { * first-run source detector can share it without pulling in either * module's heavier import graph. * + * The same contract covers every home-relative manifest path core resolves, + * not only `settings_file`: `activity_probe.dir` (the capture-health scan) + * resolves through here too, with `field` naming which manifest key a + * violation should blame. + * * @ref LLP 0045#settings_file-is-home-relative-and-a-violation-is-loud [implements]: reject an absolute settings_file rather than re-anchoring it under $HOME, and reject a relative one that climbs out of the base * @param {string} clientName * @param {string} settingsFile * @param {NodeJS.ProcessEnv | undefined} env * @param {string} homeDir + * @param {{ field?: string }} [opts] * @returns {string} * @throws {ClientSettingsPathError} when `settingsFile` is absolute, or resolves outside its base */ -export function resolveClientSettingsPath(clientName, settingsFile, env, homeDir) { +export function resolveClientSettingsPath(clientName, settingsFile, env, homeDir, opts = {}) { + const field = opts.field ?? 'settings_file' if (path.isAbsolute(settingsFile)) { throw new ClientSettingsPathError( - `client '${clientName}' declares an absolute settings_file '${settingsFile}'; ` + - "settings_file must be relative to $HOME (e.g. '.codex/config.toml')", + `client '${clientName}' declares an absolute ${field} '${settingsFile}'; ` + + `${field} must be relative to $HOME (e.g. '.codex/config.toml')`, { code: 'settings_file_absolute' } ) } @@ -85,9 +92,9 @@ export function resolveClientSettingsPath(clientName, settingsFile, env, homeDir const override = env?.[envKey] if (typeof override === 'string' && override.length > 0) { const parts = settingsFile.split('/') - return withinBase(clientName, settingsFile, override, path.join(override, ...parts.slice(1))) + return withinBase(clientName, settingsFile, override, path.join(override, ...parts.slice(1)), field) } - return withinBase(clientName, settingsFile, homeDir, path.join(homeDir, ...settingsFile.split('/'))) + return withinBase(clientName, settingsFile, homeDir, path.join(homeDir, ...settingsFile.split('/')), field) } /** @@ -117,16 +124,17 @@ export function resolveClientSettingsPath(clientName, settingsFile, env, homeDir * @param {string} settingsFile the declared value, named in the error rather than the resolved path * @param {string} base * @param {string} joined + * @param {string} field * @returns {string} * @throws {ClientSettingsPathError} when `joined` falls outside `base` */ -function withinBase(clientName, settingsFile, base, joined) { +function withinBase(clientName, settingsFile, base, joined, field) { const root = path.resolve(base) const target = path.resolve(joined) if (target !== root && !target.startsWith(root + path.sep)) { throw new ClientSettingsPathError( - `client '${clientName}' declares a settings_file '${settingsFile}' that resolves outside ` + - `'${root}'; settings_file must stay under the client's config home`, + `client '${clientName}' declares a ${field} '${settingsFile}' that resolves outside ` + + `'${root}'; ${field} must stay under the client's config home`, { code: 'settings_file_escapes_base' } ) } diff --git a/src/core/daemon/status.js b/src/core/daemon/status.js index d8851833..321161d4 100644 --- a/src/core/daemon/status.js +++ b/src/core/daemon/status.js @@ -51,7 +51,7 @@ import { /** * @import { HypAwareV2Config, PluginConfigInstance } from '../../../hypaware-plugin-kernel-types.js' * @import { ClientActionStatus, ConfigControlStatus, ConfigValidationError } from '../../../src/core/config/types.js' - * @import { ClientActionReport, ClientActionsReport, ClientAttachReport, CollectStatusOptions, DaemonStatus, DroppedUpstreamAttribution, HypAwareStatusReport, ProxyTrustReport, RecentEntrypoint, ServiceState, SinkSnapshot, SourceSnapshot, StatusDiagnostic } from '../../../src/core/daemon/types.js' + * @import { CaptureHealthReport, ClientActionReport, ClientActionsReport, ClientAttachReport, CollectStatusOptions, DaemonStatus, DroppedUpstreamAttribution, HypAwareStatusReport, ProxyTrustReport, RecentEntrypoint, ServiceState, SinkSnapshot, SourceSnapshot, StatusDiagnostic } from '../../../src/core/daemon/types.js' * @import { Dirent } from 'node:fs' * @import { ClientDescriptor, LoadedManifest, PluginCatalog } from '../../../src/core/types.js' * @import { FolderAskMode } from '../../../src/core/usage-policy/types.js' @@ -541,6 +541,111 @@ export function resolveLiveGatewayEndpointFromStatus({ stateRoot }) { return endpointFromListen(`${details.host}:${details.port}`) } +/** + * Resolve a named listener source's live bound `listen_port` from the on-disk + * daemon status snapshot, behind the same daemon-liveness gate as + * {@link resolveLiveGatewayEndpointFromStatus}: a stale snapshot from a dead + * daemon is never handed back, and no port is ever fabricated. + * + * The generic sibling of the gateway resolver above, for sources that publish + * `details.listen_port` (the OTLP receiver, the Claude telemetry listener). + * The first consumer is `hyp attach claude` in `otel` mode: only the running + * daemon knows which port the listener actually bound (its configured default, + * or the ephemeral fallback when that port was taken), so the endpoint attach + * writes must come from here whenever a daemon is up. + * + * @param {{ stateRoot: string, sourceName: string }} args + * @returns {number | undefined} + */ +export function resolveLiveSourceListenPortFromStatus({ stateRoot, sourceName }) { + const list = liveStatusSources(stateRoot) + if (!list) return undefined + const source = list.find((s) => s && s.name === sourceName) + const details = sourceDetails(source) + const port = details?.listen_port + if (typeof port !== 'number' || !Number.isInteger(port) || port < 1 || port > 65535) { + return undefined + } + return port +} + +/** + * Every live source advertising a named `/_hypaware/` control route in its + * status details (`control_routes`), resolved to the base URL of its bound + * listener. + * + * This is how `hyp session ignore` / `unignore` finds the recorders beyond + * the gateway: a recorder that hosts the route says so in its own status + * details, so the verb stays client-agnostic and a listener that is not + * running (absent from a live snapshot, or no live daemon at all) is simply + * not addressed - it is recording nothing, so there is nothing to notify. + * The gateway itself is NOT discovered here; its endpoint has its own, + * richer resolution (`status.json` plus the pinned `listen` fallback). + * + * @ref LLP 0256#cli-posts-to-both [implements]: the CLI addresses every + * listener that offers the route; offering is advertised, never guessed + * @param {{ stateRoot: string, route: string }} args + * @returns {Array<{ source: string, endpoint: string }>} + */ +export function resolveLiveControlRouteEndpointsFromStatus({ stateRoot, route }) { + const list = liveStatusSources(stateRoot) + if (!list) return [] + /** @type {Array<{ source: string, endpoint: string }>} */ + const out = [] + for (const source of list) { + if (!source || typeof source.name !== 'string') continue + const details = sourceDetails(source) + const routes = details?.control_routes + if (!Array.isArray(routes) || !routes.includes(route)) continue + const port = details?.listen_port + if (typeof port !== 'number' || !Number.isInteger(port) || port < 1 || port > 65535) continue + const host = typeof details?.listen_host === 'string' && details.listen_host.length > 0 + ? details.listen_host + : '127.0.0.1' + const endpoint = endpointFromListen(`${host}:${port}`) + if (endpoint) out.push({ source: source.name, endpoint }) + } + return out +} + +/** + * The liveness-gated snapshot read shared by the resolvers above: a live pid + * and a readable status file, or nothing. A `status.json` outlives its + * daemon, so a bound port in it proves nothing without a living process + * behind the pid file. + * + * @param {string} stateRoot + * @returns {SourceSnapshot[] | undefined} + */ +function liveStatusSources(stateRoot) { + let pidEntry + try { + pidEntry = readPidFile(stateRoot) + } catch { + return undefined + } + if (!pidEntry || !processIsAlive(pidEntry.pid)) return undefined + + /** @type {DaemonStatus | null} */ + let status + try { + status = readStatusFile(stateRoot) + } catch { + return undefined + } + return Array.isArray(status?.sources) ? status.sources : [] +} + +/** + * @param {SourceSnapshot | undefined} source + * @returns {Record | undefined} + */ +function sourceDetails(source) { + return source && typeof source.details === 'object' && source.details !== null + ? /** @type {Record} */ (source.details) + : undefined +} + /* ---------- Phase 8: top-level status collector ---------- */ /** @@ -972,6 +1077,8 @@ export async function collectHypAwareStatus(opts = {}) { } /** @type {ClientAttachReport[]} */ const clients = [] + /** @type {CaptureHealthReport[]} */ + const captureHealth = [] const clientDescriptors = catalog?.clientDescriptors ?? new Map() for (const [clientName, descriptor] of clientDescriptors) { const configured = activePlugins.includes(descriptor.plugin) @@ -999,6 +1106,11 @@ export async function collectHypAwareStatus(opts = {}) { ...(probe.settingsPath ? { settingsPath: probe.settingsPath } : {}), ...(probe.version !== undefined ? { version: probe.version } : {}), ...(probe.port !== undefined ? { port: probe.port } : {}), + ...(probe.mode !== undefined ? { mode: probe.mode } : {}), + // `--json` only, like `version` and `port`: an operator debugging a + // silent otel capture needs to see where the client is actually pointed, + // and it is the field `client_telemetry_stale` below reasons about. + ...(probe.telemetryPort !== undefined ? { telemetryPort: probe.telemetryPort } : {}), ...(probe.error !== undefined ? { error: probe.error } : {}), }) // Deliberately ungated by `attachable`, unlike the two derived-state @@ -1069,6 +1181,124 @@ export async function collectHypAwareStatus(opts = {}) { repair: [`hyp detach --client ${clientName}`], }) } + + // ----- capture health (LLP 0262 open question 1's duty) ----- + // On the otel path capture is best-effort: a stale endpoint, a down + // daemon, or upstream event drift all fail into the same silence, with + // every other line here healthy. This holds the client's own file trail + // (fresh, probed off $HOME like the attach marker was) against the last + // event the listener recorded (from status.json - deliberately NOT + // liveness-gated, the LLP 0164 argument: "last seen at T" survives its + // daemon, and the dead-daemon case is precisely the gap to surface). + // Gated on `configured` because an otel marker with no enabled plugin is + // `client_attached_not_configured`'s finding above, where the repair is a + // detach rather than a capture fix. + // @ref LLP 0257#status-and-health [implements]: last event seen vs last transcript activity, answered without a dataset or cache read + if (configured && probe.attached && probe.mode === 'otel') { + const snapshots = Array.isArray(daemonStatusFile?.sources) ? daemonStatusFile.sources : [] + const owned = snapshots.filter((s) => s && s.plugin === descriptor.plugin) + // The plugin's listener source advertises itself by carrying the + // `last_event_at` detail (null before the first event), the same + // self-advertisement pattern as `control_routes`. + const listenerSnap = owned.find((s) => { + const details = sourceDetails(s) + return !!details && 'last_event_at' in details + }) ?? owned[0] + const listenerDetails = sourceDetails(listenerSnap) + const lastEventAt = typeof listenerDetails?.last_event_at === 'string' + ? listenerDetails.last_event_at + : null + + // ----- telemetry endpoint drift ----- + // The `otel` counterpart of `client_attach_stale` above, which compares + // the marker's port against the *gateway* and so watches an address this + // mode never uses. Attach writes one endpoint into the client's settings + // and nothing rewrites it afterwards, while the listener may since have + // bound elsewhere - it falls back to an ephemeral port when its default + // is taken (LLP 0114 §ephemeral-fallback), and an attach that ran with no + // live daemon could only write the default in the first place. The client + // then POSTs its telemetry, prompts and responses included, at whatever + // process holds the port it was told about, and every other line here + // stays healthy. `capture_gap` below eventually notices the silence, but + // only after fifteen minutes of transcript activity and without naming + // the cause; this comparison is already on disk and is exact. + // + // Liveness-gated, unlike `last_event_at` right above: "the listener was + // last bound to X" is not a claim a dead daemon's snapshot can support, + // and a restart is exactly what moves the port back. + // @ref LLP 0114#fallback-is-visible [implements]: a listener that came up on its ephemeral fallback is visible in status, not only in a boot log line - here through the client left pointing at the port it vacated + // @ref LLP 0086#status-drift-diagnostic [implements]: the same warn-and-name-the-repair shape, against the port this attach mode actually writes + const boundTelemetryPort = daemon.running && typeof listenerDetails?.listen_port === 'number' + ? listenerDetails.listen_port + : undefined + if ( + probe.telemetryPort !== undefined && + boundTelemetryPort !== undefined && + Number.isInteger(boundTelemetryPort) && + probe.telemetryPort !== boundTelemetryPort + ) { + diagnostics.push({ + severity: 'warning', + kind: 'client_telemetry_stale', + message: `${clientName} exports its telemetry to port ${probe.telemetryPort} but the listener is bound to port ${boundTelemetryPort} - nothing it sends is being captured, and whatever holds port ${probe.telemetryPort} is receiving it; run 'hyp attach --client ${clientName}' to re-point it`, + repair: [ + `hyp attach --client ${clientName}`, + `start a fresh ${clientName} session - the settings env applies at launch`, + ], + }) + } + const lastTranscriptActivityAt = + (await probeClientActivityFromDescriptor({ descriptor, homeDir, env })) ?? null + const attachedAt = probe.attachedAt ?? null + // Live daemon only, deliberately. A dead daemon's snapshot still carries + // the moment its listener started, but that moment stopped bounding + // anything when the process ended, and the dead-daemon gap is the one + // this line most needs to keep reporting. + const listenerStartedAt = daemon.running && typeof listenerDetails?.listener_started_at === 'string' + ? listenerDetails.listener_started_at + : null + const verdict = assessCaptureHealth({ + lastEventAt, + lastTranscriptActivityAt, + attachedAt, + listenerStartedAt, + }) + captureHealth.push({ + client: clientName, + plugin: descriptor.plugin, + source: listenerSnap?.name ?? null, + lastEventAt, + lastTranscriptActivityAt, + attachedAt, + listenerStartedAt, + gapMs: verdict.gapMs, + state: verdict.state, + }) + if (verdict.state === 'gap' && verdict.severity !== undefined) { + // Escalates to a degrading `error` past CAPTURE_GAP_ERROR_MS, unlike + // the attach diagnostics above: a not-yet-attached install is merely + // unfinished, but an attached one silently losing sessions is the + // failure this line exists to make loud. + const gapText = formatGapDuration(verdict.gapMs) + const message = lastEventAt !== null + ? `${clientName} is otel-attached, but its transcripts stayed active ${gapText} past the last telemetry event - those sessions are not being captured` + // "past the point capture should have been running" rather than + // "after the attach": with a live listener the baseline is whichever + // of the attach and the listener's own start is newer, so naming the + // attach would be wrong exactly when a restart moved the baseline. + : `${clientName} is otel-attached, but no telemetry has arrived and its transcripts show activity ${gapText} past the point capture should have been running - those sessions are not being captured` + diagnostics.push({ + severity: verdict.severity, + kind: 'capture_gap', + message, + repair: [ + 'hyp daemon restart # the telemetry listener runs in the daemon', + `hyp attach --client ${clientName} # rewrites the telemetry env block with the live listener port`, + `start a fresh ${clientName} session - the settings env applies at launch`, + ], + }) + } + } } // ----- client sync split (LLP 0188 #never-silent) ----- @@ -1235,7 +1465,10 @@ export async function collectHypAwareStatus(opts = {}) { // client-action (e.g. backfill-on-join) is likewise excluded. It has // its own status line but never flips `overall` (LLP 0041 // §failure-is-surfaced-not-fatal); note it is not even a diagnostic, so - // it cannot reach this computation. + // it cannot reach this computation. A `capture_gap` that escalated to + // `error` severity degrades through the severity rule below by design + // (LLP 0262 open question 1): silent session loss is an outage, not an + // unfinished setup. const degradingKinds = new Set(['config_missing', 'config_unreadable']) const overall = diagnostics.some((d) => d.severity === 'error') ? 'degraded' @@ -1264,6 +1497,7 @@ export async function collectHypAwareStatus(opts = {}) { usagePolicy, firstSyncHoldDeadline, recentEntrypoints, + captureHealth, proxyTrust, } } @@ -1578,6 +1812,42 @@ function readRetention(config) { return { days: DEFAULT_RETENTION_DAYS, source: 'default' } } +/** + * The port an attach marker's managed `OTEL_EXPORTER_OTLP_ENDPOINT` names. + * + * This is not {@link probeClientAttachFromDescriptor}'s `port`, which records + * the gateway. An `otel`-mode client sends nothing to the gateway: its whole + * capture path is this one endpoint, written into the client's settings once + * at attach and never revisited. So it is the value a drift check has to + * compare, and taking it from `managed.env` - the live env block attach wrote + * and detach restores - means the check reads the address the client is + * actually using rather than a parallel field that could disagree with it. + * + * Anything that is not a well-formed loopback-shaped `http(s)://host:port` + * with an in-range port reads as absent: the marker is a file a hand edit + * reaches, and a diagnostic built on a guess is worse than no diagnostic. + * + * @param {Record} markerObj + * @returns {number | undefined} + */ +function markerTelemetryPort(markerObj) { + const managed = markerObj.managed + if (!isPlainObject(managed)) return undefined + const env = managed.env + if (!isPlainObject(env)) return undefined + const endpoint = env.OTEL_EXPORTER_OTLP_ENDPOINT + if (typeof endpoint !== 'string' || endpoint.length === 0) return undefined + let parsed + try { + parsed = new URL(endpoint) + } catch { + return undefined + } + const port = Number(parsed.port) + if (!Number.isInteger(port) || port < 1 || port > 65535) return undefined + return port +} + /** * Probe on-disk client settings using the descriptor's attach_probe * definition. Supports JSON (marker key lookup) and TOML (header string @@ -1592,7 +1862,7 @@ function readRetention(config) { * * @ref LLP 0045#settings_file-is-home-relative-and-a-violation-is-loud [implements]: an unresolvable settings_file is an error result, not a silent not-attached * @param {{ descriptor: ClientDescriptor, homeDir: string, env?: NodeJS.ProcessEnv }} args - * @returns {Promise<{ attached: boolean, settingsPath?: string, version?: string, port?: string, error?: string }>} + * @returns {Promise<{ attached: boolean, settingsPath?: string, version?: string, port?: string, mode?: string, attachedAt?: string, telemetryPort?: number, error?: string }>} */ export async function probeClientAttachFromDescriptor({ descriptor, homeDir, env }) { if (!homeDir || !descriptor.attachProbe) return { attached: false } @@ -1617,11 +1887,23 @@ export async function probeClientAttachFromDescriptor({ descriptor, homeDir, env const marker = /** @type {Record} */ (parsed)[probe.marker_key] if (!marker || typeof marker !== 'object') return { attached: false, settingsPath } const markerObj = /** @type {Record} */ (marker) + const telemetryPort = markerTelemetryPort(markerObj) return { attached: true, settingsPath, version: typeof markerObj.version === 'string' ? markerObj.version : undefined, port: typeof markerObj.port === 'number' ? String(markerObj.port) : undefined, + // The marker's `mode` / `attached_at`, absent on markers that predate + // them: mode is what gates the capture-health section, and the attach + // timestamp is its baseline for a listener that has seen nothing yet. + ...(typeof markerObj.mode === 'string' ? { mode: markerObj.mode } : {}), + ...(typeof markerObj.attached_at === 'string' ? { attachedAt: markerObj.attached_at } : {}), + // Where the client's own exporter is pointed, which for an `otel` + // attach is the only address capture depends on. Read off + // `managed.env` rather than added as a second marker field, so it is + // literally the value the client is using and cannot fall out of step + // with it. + ...(telemetryPort !== undefined ? { telemetryPort } : {}), } } @@ -1661,6 +1943,176 @@ export async function probeClientAttachFromDescriptor({ descriptor, homeDir, env } } +/** + * Ceiling on how deep the activity-probe walk descends below the declared + * directory. Claude transcripts sit two levels down + * (`projects//.jsonl`) and subagent transcripts four + * (`projects///subagents/agent-*.jsonl`); the cap exists so a + * manifest pointing at a pathological tree bounds the probe instead of the + * probe walking it to the bottom. + */ +const MAX_ACTIVITY_PROBE_DEPTH = 5 + +/** + * When this client last left a file behind: the newest matching mtime under + * the descriptor's `activity_probe.dir`, as an ISO timestamp. + * + * This is the transcript half of the capture-health comparison, probed fresh + * on every `hyp status` run rather than read from `status.json`, because the + * moment it matters most is a daemon that has been down while the user + * worked - exactly when nothing was alive to record it. It stats file + * metadata only, never opens a file, and is best-effort like every probe in + * this collector: any failure reads as `undefined` (no claim), never as a + * fabricated timestamp. + * + * @ref LLP 0257#status-and-health [implements]: the last-transcript-activity side of the capture-health line + * @param {{ descriptor: ClientDescriptor, homeDir: string, env?: NodeJS.ProcessEnv }} args + * @returns {Promise} + */ +export async function probeClientActivityFromDescriptor({ descriptor, homeDir, env }) { + const probe = descriptor.activityProbe + if (!probe || !homeDir) return undefined + /** @type {string} */ + let dirPath + try { + dirPath = resolveClientSettingsPath(descriptor.name, probe.dir, env, homeDir, { + field: 'activity_probe.dir', + }) + } catch { + return undefined + } + const newest = await newestMtimeMs(dirPath, probe.file_suffix, MAX_ACTIVITY_PROBE_DEPTH) + return newest === undefined ? undefined : new Date(newest).toISOString() +} + +/** + * Newest mtime (epoch ms) of any matching regular file under `dir`, walked + * to `depth` levels. Symlinks are not followed and every fs error skips the + * entry: a probe that cannot read a corner of the tree still answers from + * the rest of it. + * + * @param {string} dir + * @param {string | undefined} suffix + * @param {number} depth + * @returns {Promise} + */ +async function newestMtimeMs(dir, suffix, depth) { + /** @type {Dirent[]} */ + let entries + try { + entries = await fsp.readdir(dir, { withFileTypes: true }) + } catch { + return undefined + } + /** @type {number | undefined} */ + let newest + for (const entry of entries) { + const full = path.join(dir, entry.name) + if (entry.isDirectory()) { + if (depth <= 1) continue + const nested = await newestMtimeMs(full, suffix, depth - 1) + if (nested !== undefined && (newest === undefined || nested > newest)) newest = nested + } else if (entry.isFile()) { + if (suffix !== undefined && !entry.name.endsWith(suffix)) continue + try { + const stat = await fsp.stat(full) + if (newest === undefined || stat.mtimeMs > newest) newest = stat.mtimeMs + } catch { /* raced deletion or unreadable file: skip */ } + } + } + return newest +} + +/** + * Client activity newer than the capture baseline by more than this is a + * capture gap worth a diagnostic. Under working capture the two move in near + * lockstep - Claude Code appends the transcript and flushes the exporter on + * the same turns, seconds apart - so the threshold only has to clear flush + * cadence and batch timing, and fifteen minutes clears them by an order of + * magnitude while still catching a broken path within the same sitting. + */ +export const CAPTURE_GAP_WARNING_MS = 15 * 60_000 + +/** + * Past this the gap severity escalates to `error`, which degrades `overall`: + * two hours of transcript activity with no telemetry is a whole working + * session lost, not a timing artifact. The escalation is the "visible + * instead of discovered at report time" duty of LLP 0262 open question 1 - + * best-effort delivery was accepted on the condition that a silent gap + * cannot stay silent. + */ +export const CAPTURE_GAP_ERROR_MS = 2 * 3_600_000 + +/** + * Judge one otel-attached client's capture gap. Pure, so the threshold + * contract is unit-testable without a filesystem. + * + * The baseline is the newest of three moments capture could be measured from: + * the last event seen, the attach timestamp, and the running listener's own + * start. Activity older than the attach proves nothing about the otel path + * (the usual shape right after a migration from proxy attach, where months of + * transcripts predate the first possible event), and a listener that has seen + * nothing at all is measured from the attach instead. No baseline at all - no + * events, no listener, and an unreadable attach time - reads as `ok`, because + * a gap claim needs a moment capture was supposed to start. + * + * `listenerStartedAt` is the third because the listener's `lastEventAt` lives + * only in its process: every daemon restart republishes `last_event_at: null` + * however long capture has been healthy, and without this the baseline would + * fall back to an attach timestamp that can be weeks old. A machine attached a + * month ago and used an hour ago would then report a month-long gap - severity + * `error`, degrading `overall` - immediately after a routine + * `hyp daemon restart`, which is itself the first repair `capture_gap` prints. + * The caller passes it ONLY for a live daemon: on a dead one the last daemon's + * start says nothing about now, and the growing gap is precisely the thing to + * surface. + * + * @ref LLP 0257#status-and-health [implements]: the gap threshold and its severity + * @param {{ lastEventAt?: string | null, lastTranscriptActivityAt?: string | null, attachedAt?: string | null, listenerStartedAt?: string | null }} args + * @returns {{ state: 'ok' | 'gap', gapMs: number, severity?: 'warning' | 'error' }} + */ +export function assessCaptureHealth({ lastEventAt, lastTranscriptActivityAt, attachedAt, listenerStartedAt }) { + const transcriptMs = parseIsoMs(lastTranscriptActivityAt) + if (transcriptMs === undefined) return { state: 'ok', gapMs: 0 } + const eventMs = parseIsoMs(lastEventAt) + const attachedMs = parseIsoMs(attachedAt) + const listenerMs = parseIsoMs(listenerStartedAt) + if (eventMs === undefined && attachedMs === undefined && listenerMs === undefined) { + return { state: 'ok', gapMs: 0 } + } + const baseline = Math.max(eventMs ?? -Infinity, attachedMs ?? -Infinity, listenerMs ?? -Infinity) + const gapMs = Math.max(0, transcriptMs - baseline) + if (gapMs <= CAPTURE_GAP_WARNING_MS) return { state: 'ok', gapMs } + return { + state: 'gap', + gapMs, + severity: gapMs > CAPTURE_GAP_ERROR_MS ? 'error' : 'warning', + } +} + +/** + * A gap length for diagnostic prose: coarse on purpose, like + * `formatEntrypointAge`, because the message's claim is "a sitting" or "a + * day", never a precise bound. + * + * @param {number} gapMs + * @returns {string} + */ +export function formatGapDuration(gapMs) { + const minutes = Math.floor(gapMs / 60_000) + if (minutes < 60) return `${minutes}m` + const hours = Math.floor(minutes / 60) + if (hours < 48) return `${hours}h` + return `${Math.floor(hours / 24)}d` +} + +/** @param {string | null | undefined} value @returns {number | undefined} */ +function parseIsoMs(value) { + if (typeof value !== 'string') return undefined + const ms = Date.parse(value) + return Number.isNaN(ms) ? undefined : ms +} + /** * Build the plugin catalog the status surfaces read from: bundled ⊕ installed. * Best-effort, exactly as the top-level collector was: each discovery failure diff --git a/src/core/daemon/types.d.ts b/src/core/daemon/types.d.ts index 944c9afc..c5ea98f7 100644 --- a/src/core/daemon/types.d.ts +++ b/src/core/daemon/types.d.ts @@ -98,6 +98,7 @@ export type StatusDiagnosticKind = | 'daemon_loaded_no_pid' | 'client_attach_missing' | 'client_attach_stale' + | 'client_telemetry_stale' | 'client_attached_not_configured' | 'gateway_port_fallback' | 'gateway_idle_no_upstreams' @@ -106,6 +107,7 @@ export type StatusDiagnosticKind = | 'remote_config_rolled_back' | 'local_only_list_unreadable' | 'client_sync_list_unreadable' + | 'capture_gap' /** * Diagnostic surfaced by `hyp status`. Carries a severity, the @@ -223,10 +225,57 @@ export interface ClientAttachReport { version?: string /** Local gateway port the adapter routes through, when recorded. */ port?: string + /** Attach mode recorded in the marker (`base_url` / `proxy` / `otel`), when present. */ + mode?: string + /** ISO timestamp the marker records the attach at, when present. */ + attachedAt?: string + /** + * Port the marker's managed `OTEL_EXPORTER_OTLP_ENDPOINT` sends telemetry + * to, when the marker carries one. Distinct from `port` above (the + * gateway's): in `otel` mode this is the only address capture depends on, + * and it is what `client_telemetry_stale` compares against the listener's + * live bind. + */ + telemetryPort?: number /** Probe error string, when the file was unreadable. */ error?: string } +/** + * One otel-attached client's capture health: what its own file trail says it + * did (`lastTranscriptActivityAt`, the newest activity-probe mtime) held + * against what the telemetry path actually captured (`lastEventAt`, from the + * listener source's status.json details). `state` is `gap` when activity ran + * past the capture baseline by more than the threshold; the paired + * `capture_gap` diagnostic carries the severity and repair + * (LLP 0257#status-and-health). + */ +export interface CaptureHealthReport { + /** Client name (`claude`). */ + client: string + /** The plugin that owns the client and its listener source. */ + plugin: string + /** The listener source's snapshot name in status.json, or null when no daemon recorded one. */ + source: string | null + /** Last telemetry event the listener saw, or null when none is recorded. */ + lastEventAt: string | null + /** Newest activity-probe file mtime, or null when the trail is empty or unprobed. */ + lastTranscriptActivityAt: string | null + /** The attach timestamp the marker records, or null when unreadable. */ + attachedAt: string | null + /** + * When the live listener started, or null when no daemon is running. The + * third baseline the gap is measured from: `lastEventAt` is in-process + * state, so a restart republishes it as null however long capture has been + * healthy, and without this the gap would be measured from an attach that + * can be weeks old. + */ + listenerStartedAt: string | null + /** Milliseconds of activity past the capture baseline (0 when none). */ + gapMs: number + state: 'ok' | 'gap' +} + /** Service-level daemon state surfaced by `hyp status`. */ export interface ServiceState { /** Service file present at the platform path. */ @@ -355,6 +404,16 @@ export interface HypAwareStatusReport { * and reads no cache, so this is the only place the answer can come from. */ recentEntrypoints: RecentEntrypoint[] + /** + * Capture health for every otel-attached client (LLP 0257#status-and-health, + * the RFC 0262 open-question-1 duty): last event seen on the telemetry path + * vs the client's own last activity. Empty when no configured client is + * otel-attached, so the pre-otel surface is unchanged. Like + * `recentEntrypoints` this reads status.json without a liveness gate: a + * dead daemon's stale `lastEventAt` is exactly the evidence a capture gap + * is made of. + */ + captureHealth: CaptureHealthReport[] /** * Proxy-mode trust state (LLP 0237, LLP 0239). Null whenever the question * does not apply: a non-darwin host (both mechanisms are macOS-only, LLP diff --git a/hypaware-core/plugins-workspace/otel/src/server.js b/src/core/otlp/server.js similarity index 57% rename from hypaware-core/plugins-workspace/otel/src/server.js rename to src/core/otlp/server.js index 9b271b22..c393f188 100644 --- a/hypaware-core/plugins-workspace/otel/src/server.js +++ b/src/core/otlp/server.js @@ -3,46 +3,73 @@ import http from 'node:http' import zlib from 'node:zlib' +import { isControlPath } from '../control/session_ignore.js' + /** - * @import { OtlpReceiveHandler, OtlpSignal } from './types.js' + * @import { OtlpJsonServerOptions, OtlpSignal } from '../../../src/core/otlp/types.js' */ const JSON_CT = { 'Content-Type': 'application/json' } -const SIGNAL_ROUTES = /** @type {const} */ ({ +/** Path to signal, the OTLP/HTTP standard routes. */ +const SIGNAL_ROUTES = /** @type {Record} */ ({ '/v1/logs': 'logs', '/v1/traces': 'traces', '/v1/metrics': 'metrics', }) +const ALL_SIGNALS = /** @type {readonly OtlpSignal[]} */ (['logs', 'traces', 'metrics']) + +/** The success envelope OTLP requires per signal: nothing was rejected. */ const EMPTY_PARTIAL_SUCCESS = { logs: { partialSuccess: { rejectedLogRecords: 0 } }, traces: { partialSuccess: { rejectedSpans: 0 } }, metrics: { partialSuccess: { rejectedDataPoints: 0 } }, } +// @ref LLP 0257#registration [implements]: one OTLP http/json server, hosted by more than one plugin; payload interpretation stays behind the handler /** - * Create the OTLP/HTTP listener. The handler is invoked once per - * decoded request with `{ signal, data, payloadBytes }`. Errors thrown - * by the handler bubble up as HTTP 500; the caller (the source's - * `start`) is responsible for wrapping that path in an `otel.receive` - * span and translating exception types to `error_kind` attributes. + * Create an OTLP/HTTP listener. The handler is invoked once per decoded + * request with `{ signal, data, payloadBytes }`. Errors thrown by the + * handler bubble up as HTTP 500; the caller (the source's `start`) is + * responsible for wrapping that path in a receive span and translating + * exception types to `error_kind` attributes. * - * Only OTLP/JSON is accepted in this pass: an OTLP/protobuf decoder - * chain was left out of V1 and can be added later without changing - * the request handler shape. + * Only OTLP/JSON is accepted: an OTLP/protobuf decoder chain was left + * out of V1 and can be added later without changing the handler shape. * - * @param {OtlpReceiveHandler} handler + * Everything this server knows is transport: routing, content type, + * content encoding, and the `partialSuccess` envelope. It never reads + * inside `data`, so a second plugin can host a listener with entirely + * different payload semantics. + * + * @param {OtlpJsonServerOptions} options * @returns {http.Server} */ -export function createOtlpServer(handler) { +export function createOtlpJsonServer(options) { + const { name, handler } = options + const served = new Set(options.signals ?? ALL_SIGNALS) + return http.createServer(async (req, res) => { const url = new URL(req.url ?? '/', `http://${req.headers.host || 'localhost'}`) const route = url.pathname + // The reserved `/_hypaware/` prefix is a LOCAL control surface, exactly + // as it is on the gateway proxy: short-circuited before any OTLP + // routing, so a control request is never read as an export and an + // OTLP path can never shadow a control route. Hosts that register no + // handler keep the old behavior (the paths fall through and 404 as + // unknown OTLP routes below). + // @ref LLP 0256#control-route-on-listener [implements]: the listener serves + // the same control surface the proxy serves, through the same handler + if (isControlPath(route) && typeof options.onControlRequest === 'function') { + options.onControlRequest(req, res, url) + return + } + if (req.method === 'GET' && route === '/') { res.writeHead(200, { 'Content-Type': 'text/plain; charset=utf-8' }) - res.end('hypaware/otel OTLP listener\n') + res.end(`${name} OTLP listener\n`) return } @@ -51,10 +78,8 @@ export function createOtlpServer(handler) { return } - const signal = /** @type {OtlpSignal | undefined} */ ( - /** @type {Record} */ (SIGNAL_ROUTES)[route] - ) - if (!signal) { + const signal = SIGNAL_ROUTES[route] + if (!signal || !served.has(signal)) { respondJsonError(res, 404, 5, 'Not found') return } @@ -125,15 +150,17 @@ function respondJsonError(res, httpStatus, code, message) { /** * Listen on `host:port` and resolve with the actually bound `{ host, port }`. - * Wraps the awkward `server.listen` callback / `address()` shape so the - * source's `start` reads as a straight-line coroutine. + * Wraps the awkward `server.listen` callback / `address()` shape so a + * source's `start` reads as a straight-line coroutine, and so a listener + * asking for port `0` learns the port it actually got. * * @param {http.Server} server * @param {string} host * @param {number} port + * @param {string} [name] listener name, used only in the failure message * @returns {Promise<{ host: string, port: number }>} */ -export function listenAndResolve(server, host, port) { +export function listenAndResolve(server, host, port, name = 'hypaware') { return new Promise((resolve, reject) => { /** @param {Error} err */ function onError(err) { @@ -146,7 +173,7 @@ export function listenAndResolve(server, host, port) { if (addr && typeof addr === 'object') { resolve({ host: addr.address, port: addr.port }) } else { - reject(new Error('hypaware/otel: server.address() returned no AddressInfo')) + reject(new Error(`${name}: server.address() returned no AddressInfo`)) } } server.once('error', onError) diff --git a/src/core/otlp/types.d.ts b/src/core/otlp/types.d.ts new file mode 100644 index 00000000..533eb2fe --- /dev/null +++ b/src/core/otlp/types.d.ts @@ -0,0 +1,35 @@ +import type { IncomingMessage, ServerResponse } from 'node:http' + +/** The three OTLP signals the shared http/json listener routes. */ +export type OtlpSignal = 'logs' | 'traces' | 'metrics' + +/** One decoded OTLP/JSON request, handed to the hosting plugin. */ +export interface OtlpRequest { + signal: OtlpSignal + data: unknown + payloadBytes: number +} + +/** + * The hosting plugin's side of the seam. Payload interpretation lives + * behind this handle, never in the shared server. + */ +export interface OtlpReceiveHandler { + handle(req: OtlpRequest): Promise +} + +/** Options for `createOtlpJsonServer`. */ +export interface OtlpJsonServerOptions { + /** Listener name, used in the `GET /` banner and in bind errors. */ + name: string + /** Invoked once per decoded request. */ + handler: OtlpReceiveHandler + /** Signal paths to serve. Defaults to logs, traces and metrics. */ + signals?: readonly OtlpSignal[] + /** + * Serves the reserved `/_hypaware/` local control surface, short-circuited + * before OTLP routing. The handler owns the request lifecycle (body and + * response). Absent, control paths fall through as unknown OTLP routes. + */ + onControlRequest?: (req: IncomingMessage, res: ServerResponse, url: URL) => void +} diff --git a/src/core/plugin_catalog.js b/src/core/plugin_catalog.js index 696384b5..3cd9f46a 100644 --- a/src/core/plugin_catalog.js +++ b/src/core/plugin_catalog.js @@ -89,6 +89,19 @@ export function buildPluginCatalog(bundledManifests, installedManifests = []) { (v) => typeof v === 'string' && v.length > 0 ) } + // A probe with no readable `dir` is dropped here rather than + // downstream: an accepted-but-empty probe would report "no + // activity" for a client that is active, which reads as + // capture health it never measured. + const activity = client.activity_probe + if (activity && typeof activity.dir === 'string' && activity.dir.length > 0) { + descriptor.activityProbe = { + dir: activity.dir, + ...(typeof activity.file_suffix === 'string' && activity.file_suffix.length > 0 + ? { file_suffix: activity.file_suffix } + : {}), + } + } // A launch spec that cannot carry the question is dropped here // rather than downstream: an accepted-but-mute spec starts the // client with no prompt, which reads as the feature working. diff --git a/src/core/types.d.ts b/src/core/types.d.ts index 8f961f5a..5e39d0b1 100644 --- a/src/core/types.d.ts +++ b/src/core/types.d.ts @@ -1,5 +1,6 @@ import type { PickerDetectProbe, + PluginActivityProbeManifest, PluginAttachProbeManifest, PluginContributionManifest, PluginClientLaunchManifest, @@ -40,6 +41,13 @@ export interface ClientDescriptor { * (LLP 0198#split). */ launch?: PluginClientLaunchManifest + /** + * Where this client's own activity leaves a file trail, from + * `contributes.client.activity_probe`. Read by the `hyp status` + * capture-health probe (LLP 0257#status-and-health); absent for a + * client whose activity leaves no scannable trail. + */ + activityProbe?: PluginActivityProbeManifest } /** diff --git a/test/core/capture-spool.test.js b/test/core/capture-spool.test.js new file mode 100644 index 00000000..075531da --- /dev/null +++ b/test/core/capture-spool.test.js @@ -0,0 +1,135 @@ +// @ts-check + +/** + * The capture spool as core knows it: where it lives, which directories a + * settings-file marker may point a sweep at, and what emptying one does. + * + * @ref LLP 0253#spool-location [tests]: `/spool/`, which is + * what purge and detach can find without being told + * @ref LLP 0253#purge-and-detach-sweep [tests]: emptying removes the contents + * and leaves the directory + */ + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { + captureSpoolRoot, + isCaptureSpoolDir, + sweepCaptureSpool, +} from '../../src/core/capture_spool.js' +import { claudeBodySpoolDir } from '../../hypaware-core/plugins-workspace/claude/src/telemetry/spool.js' + +/** @param {string} prefix */ +function tmpDir(prefix) { + return fs.mkdtemp(path.join(os.tmpdir(), `hyp-spool-${prefix}-`)) +} + +test('captureSpoolRoot: the claude body spool is a child of the shared root', () => { + const home = path.join(os.tmpdir(), 'hyp-home-fixture') + assert.equal(captureSpoolRoot(home), path.join(home, 'spool')) + assert.equal(path.dirname(claudeBodySpoolDir(home)), captureSpoolRoot(home)) + assert.equal(isCaptureSpoolDir(claudeBodySpoolDir(home), home), true) +}) + +test('isCaptureSpoolDir: only a direct child of /spool qualifies', () => { + const home = '/hyp/home' + assert.equal(isCaptureSpoolDir('/hyp/home/spool/claude-bodies', home), true) + // The root itself, a nested path, and anything outside are all refused: the + // value comes off a settings file a hand edit can reach. + assert.equal(isCaptureSpoolDir('/hyp/home/spool', home), false) + assert.equal(isCaptureSpoolDir('/hyp/home/spool/claude-bodies/deeper', home), false) + assert.equal(isCaptureSpoolDir('/hyp/home/cache', home), false) + assert.equal(isCaptureSpoolDir('/etc', home), false) + // A traversal spelling normalizes before the test, so it lands outside. + assert.equal(isCaptureSpoolDir('/hyp/home/spool/../../../etc', home), false) + assert.equal(isCaptureSpoolDir('relative/spool/claude-bodies', home), false) + assert.equal(isCaptureSpoolDir(undefined, home), false) + assert.equal(isCaptureSpoolDir(42, home), false) +}) + +test('sweepCaptureSpool: removes every file, keeps the directories, reports counts', async () => { + const home = await tmpDir('sweep') + try { + const dir = claudeBodySpoolDir(home) + const nested = path.join(dir, 'nested') + await fs.mkdir(nested, { recursive: true }) + await fs.writeFile(path.join(dir, 'a.json'), '0123456789') + await fs.writeFile(path.join(dir, 'b.json'), '01234') + await fs.writeFile(path.join(nested, 'c.json'), '012') + + const swept = await sweepCaptureSpool(dir) + assert.equal(swept.filesRemoved, 3) + assert.equal(swept.bytesRemoved, 18) + assert.equal(swept.failed, 0) + + // The directory survives: the client was told this path at attach and is + // not asked again. + assert.deepEqual(await fs.readdir(dir), ['nested']) + assert.deepEqual(await fs.readdir(nested), []) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +test('sweepCaptureSpool: an absent spool is a no-op, not a failure', async () => { + const home = await tmpDir('absent') + try { + const swept = await sweepCaptureSpool(claudeBodySpoolDir(home)) + assert.deepEqual(swept, { filesRemoved: 0, bytesRemoved: 0, failed: 0 }) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +test('sweepCaptureSpool: a file it cannot remove is counted, not thrown', async () => { + const home = await tmpDir('failing') + try { + const dir = claudeBodySpoolDir(home) + await fs.mkdir(dir, { recursive: true }) + await fs.writeFile(path.join(dir, 'a.json'), 'x') + await fs.writeFile(path.join(dir, 'b.json'), 'yy') + + const real = fs + const failing = /** @type {any} */ ({ + readdir: real.readdir, + lstat: real.lstat, + /** @param {string} target @param {unknown} opts */ + rm: async (target, opts) => { + if (path.basename(target) === 'a.json') throw new Error('EPERM: nope') + return real.rm(target, /** @type {any} */ (opts)) + }, + }) + + const swept = await sweepCaptureSpool(dir, { fs: failing }) + assert.equal(swept.filesRemoved, 1) + assert.equal(swept.bytesRemoved, 2) + assert.equal(swept.failed, 1) + assert.deepEqual(await fs.readdir(dir), ['a.json']) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +test('sweepCaptureSpool: a symlink is removed as the link, not followed', async () => { + const home = await tmpDir('symlink') + const outside = await tmpDir('symlink-target') + try { + const kept = path.join(outside, 'keep-me.txt') + await fs.writeFile(kept, 'not ours') + const dir = claudeBodySpoolDir(home) + await fs.mkdir(dir, { recursive: true }) + await fs.symlink(kept, path.join(dir, 'link.json')) + + const swept = await sweepCaptureSpool(dir) + assert.equal(swept.filesRemoved, 1) + assert.deepEqual(await fs.readdir(dir), []) + assert.equal(await fs.readFile(kept, 'utf8'), 'not ours') + } finally { + await fs.rm(home, { recursive: true, force: true }) + await fs.rm(outside, { recursive: true, force: true }) + } +}) diff --git a/test/core/client-detach-spool-sweep.test.js b/test/core/client-detach-spool-sweep.test.js new file mode 100644 index 00000000..a837764c --- /dev/null +++ b/test/core/client-detach-spool-sweep.test.js @@ -0,0 +1,157 @@ +// @ts-check + +/** + * Detach empties the raw-body spool the attach marker recorded. + * + * The undo is plugin-agnostic, so it learns the directory from the marker + * rather than computing it - which makes the path settings-file input, and the + * containment gate part of the behavior under test rather than an + * implementation detail. + * + * @ref LLP 0253#purge-and-detach-sweep [tests]: detach removes the spool + * directory's contents + * @ref LLP 0258#marker-and-spool [tests]: the recorded path is what the undo + * uses + */ + +import assert from 'node:assert/strict' +import fsp from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { detachClientFromDisk } from '../../src/core/config/client_detach_disk.js' +import { MODE_OTEL, attach } from '../../hypaware-core/plugins-workspace/claude/src/settings.js' +import { claudeBodySpoolDir } from '../../hypaware-core/plugins-workspace/claude/src/telemetry/spool.js' + +/** @import { ClientDescriptor } from '../../src/core/types.js' */ + +/** @type {ClientDescriptor} */ +const CLAUDE_DESCRIPTOR = { + plugin: /** @type {any} */ ('@hypaware/claude'), + name: 'claude', + skillDir: 'skills/claude', + attachProbe: { format: 'json', settings_file: '.claude/settings.json', marker_key: '_hypaware' }, +} + +/** A temp home carrying an `otel`-mode attach and a spool with bodies in it. */ +async function rig() { + const root = await fsp.mkdtemp(path.join(os.tmpdir(), 'hyp-detach-spool-')) + const hypHome = path.join(root, '.hyp') + const settingsPath = path.join(root, '.claude', 'settings.json') + await fsp.mkdir(path.dirname(settingsPath), { recursive: true }) + const spoolDir = claudeBodySpoolDir(hypHome) + + return { + root, + hypHome, + settingsPath, + spoolDir, + env: { HOME: root, HYP_HOME: hypHome }, + /** @param {string} [dir] */ + async stageBodies(dir = spoolDir) { + await fsp.mkdir(dir, { recursive: true }) + await fsp.writeFile(path.join(dir, 'req-1.json'), '{"messages":["a raw prompt"]}') + await fsp.writeFile(path.join(dir, 'resp-1.json'), '{"content":["a raw reply"]}') + }, + /** @param {{ spoolDir?: string }} [extra] */ + attachOtel(extra = {}) { + return attach({ + port: 18521, + version: '2.0.0', + stateFile: path.join(root, 'session-context.jsonl'), + settingsPath, + mode: MODE_OTEL, + telemetryPort: 4319, + spoolDir, + claudeVersion: '2.1.233', + ...extra, + }) + }, + detach() { + return detachClientFromDisk({ + descriptor: CLAUDE_DESCRIPTOR, + homeDir: root, + env: /** @type {any} */ ({ HOME: root, HYP_HOME: hypHome }), + }) + }, + /** @returns {Promise>} */ + async readSettings() { + return JSON.parse(await fsp.readFile(settingsPath, 'utf8')) + }, + cleanup: () => fsp.rm(root, { recursive: true, force: true }), + } +} + +test('detach empties the spool the marker recorded and keeps the directory', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + await r.attachOtel() + await r.stageBodies() + assert.equal((await fsp.readdir(r.spoolDir)).length, 2) + + const result = await r.detach() + assert.equal(result.changed, true) + assert.equal(result.warning, undefined) + assert.deepEqual(await fsp.readdir(r.spoolDir), []) +}) + +test('detach leaves a spool the marker names outside the HypAware home alone, and says so', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + const foreign = path.join(r.root, 'not-a-spool') + await r.attachOtel() + // The marker is in the user's own settings file: a hand edit can repoint it, + // and the undo must not become a delete primitive because of that. + const settings = await r.readSettings() + settings._hypaware.spool_dir = foreign + await fsp.writeFile(r.settingsPath, JSON.stringify(settings, null, 2) + '\n') + await r.stageBodies(foreign) + + const result = await r.detach() + assert.equal(result.changed, true) + assert.match(String(result.warning), /body spool outside/) + assert.equal((await fsp.readdir(foreign)).length, 2) +}) + +test('a proxy-mode marker records no spool, so detach sweeps nothing', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + // Bodies from a previous otel attach are still on disk, but this marker does + // not name them, and the undo never guesses a path. + await r.stageBodies() + await fsp.mkdir(r.hypHome, { recursive: true }) + await fsp.writeFile(path.join(r.hypHome, 'ca.pem'), '-----BEGIN CERTIFICATE-----\nfixture\n-----END CERTIFICATE-----\n') + await attach({ + port: 18521, + version: '2.0.0', + stateFile: path.join(r.root, 'session-context.jsonl'), + settingsPath: r.settingsPath, + mode: 'proxy', + caCertPath: path.join(r.hypHome, 'ca.pem'), + }) + + const result = await r.detach() + assert.equal(result.changed, true) + assert.equal((await fsp.readdir(r.spoolDir)).length, 2) +}) + +test('a marker whose undo record was damaged still sweeps its spool', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + await r.attachOtel() + await r.stageBodies() + // The `managed` record is what a hand edit (or corruption) loses first; the + // top-level `mode` and `spool_dir` survive it, and so must the sweep. + const settings = await r.readSettings() + delete settings._hypaware.managed + await fsp.writeFile(r.settingsPath, JSON.stringify(settings, null, 2) + '\n') + + const result = await r.detach() + assert.equal(result.changed, true) + assert.deepEqual(await fsp.readdir(r.spoolDir), []) +}) diff --git a/test/core/init-gateway-listen-default.test.js b/test/core/init-gateway-listen-default.test.js index d97b3aad..a4f19f97 100644 --- a/test/core/init-gateway-listen-default.test.js +++ b/test/core/init-gateway-listen-default.test.js @@ -110,6 +110,7 @@ test('the claude-and-otel-local preset leaves the gateway listen unset so the fi commands: { register() {} }, skills: { register() {} }, agents: { register() {} }, + query: { registerDataset() {} }, initPresets: { register(/** @type {any} */ p) { preset = p } }, } await activateClaude(ctx) diff --git a/test/core/init-preset-composes-graph.test.js b/test/core/init-preset-composes-graph.test.js index 77c6cba1..c6b4bb1a 100644 --- a/test/core/init-preset-composes-graph.test.js +++ b/test/core/init-preset-composes-graph.test.js @@ -60,6 +60,7 @@ async function runPreset() { commands: { register() {} }, skills: { register() {} }, agents: { register() {} }, + query: { registerDataset() {} }, initPresets: { register(/** @type {any} */ p) { preset = p } }, } await activateClaude(ctx) diff --git a/test/core/init-proxy-mode-default.test.js b/test/core/init-proxy-mode-default.test.js index 34753ea5..6c0c0c75 100644 --- a/test/core/init-proxy-mode-default.test.js +++ b/test/core/init-proxy-mode-default.test.js @@ -82,6 +82,7 @@ test('the claude-and-otel-local preset writes proxy_mode: true', async () => { commands: { register() {} }, skills: { register() {} }, agents: { register() {} }, + query: { registerDataset() {} }, initPresets: { register(/** @type {any} */ p) { preset = p } }, } await activateClaude(ctx) diff --git a/test/core/otlp-json-server.test.js b/test/core/otlp-json-server.test.js new file mode 100644 index 00000000..8ba871ba --- /dev/null +++ b/test/core/otlp-json-server.test.js @@ -0,0 +1,385 @@ +// @ts-check + +// The transport contract of the shared OTLP http/json listener. Two +// plugins host listeners on this machinery (LLP 0257 #registration), so +// the routing, content-type, content-encoding and envelope behaviour is +// pinned here rather than inside either plugin's tests. + +import test from 'node:test' +import assert from 'node:assert/strict' +import http from 'node:http' +import zlib from 'node:zlib' + +import { createOtlpJsonServer, listenAndResolve } from '../../src/core/otlp/server.js' + +/** + * @import { OtlpJsonServerOptions, OtlpRequest, OtlpSignal } from '../../src/core/otlp/types.js' + */ + +/** + * Start a listener on a dynamic loopback port and return an origin plus + * the requests the handler saw. `onRequest` lets a test make the handler + * fail. + * + * @param {{ + * name?: string, + * signals?: readonly OtlpSignal[], + * onRequest?: (req: OtlpRequest) => void, + * onControlRequest?: OtlpJsonServerOptions['onControlRequest'], + * }} [options] + */ +async function startServer(options = {}) { + /** @type {OtlpRequest[]} */ + const seen = [] + const server = createOtlpJsonServer({ + name: options.name ?? 'hypaware/test', + signals: options.signals, + onControlRequest: options.onControlRequest, + handler: { + async handle(req) { + seen.push(req) + options.onRequest?.(req) + }, + }, + }) + const bound = await listenAndResolve(server, '127.0.0.1', 0, 'hypaware/test') + return { + seen, + bound, + origin: `http://127.0.0.1:${bound.port}`, + async close() { + await new Promise((resolve, reject) => { + server.close((err) => (err ? reject(err) : resolve(undefined))) + server.closeIdleConnections?.() + server.closeAllConnections?.() + }) + }, + } +} + +/** + * POST with no `Content-Type` header at all, which `fetch` cannot express. + * + * @param {number} port + * @param {string} path + * @returns {Promise<{ status: number, body: string }>} + */ +function postWithoutContentType(port, path) { + return new Promise((resolve, reject) => { + const req = http.request({ host: '127.0.0.1', port, path, method: 'POST' }, (res) => { + /** @type {Buffer[]} */ + const chunks = [] + res.on('data', (chunk) => chunks.push(chunk)) + res.on('end', () => + resolve({ status: res.statusCode ?? 0, body: Buffer.concat(chunks).toString('utf8') }) + ) + }) + req.on('error', reject) + req.end('{}') + }) +} + +test('listenAndResolve reports the port a dynamic bind actually got', async () => { + const s = await startServer() + try { + assert.equal(s.bound.host, '127.0.0.1') + assert.ok(s.bound.port > 0) + } finally { + await s.close() + } +}) + +test('GET / answers with the listener name banner', async () => { + const s = await startServer({ name: 'hypaware/otel' }) + try { + const res = await fetch(`${s.origin}/`) + assert.equal(res.status, 200) + assert.equal(await res.text(), 'hypaware/otel OTLP listener\n') + } finally { + await s.close() + } +}) + +test('each signal route answers with its own empty partialSuccess envelope', async () => { + const s = await startServer() + try { + /** @type {[string, string][]} */ + const cases = [ + ['/v1/logs', 'rejectedLogRecords'], + ['/v1/traces', 'rejectedSpans'], + ['/v1/metrics', 'rejectedDataPoints'], + ] + for (const [route, field] of cases) { + const res = await fetch(`${s.origin}${route}`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(res.status, 200) + assert.equal(res.headers.get('content-type'), 'application/json') + assert.deepEqual(await res.json(), { partialSuccess: { [field]: 0 } }) + } + assert.deepEqual( + s.seen.map((req) => req.signal), + ['logs', 'traces', 'metrics'] + ) + } finally { + await s.close() + } +}) + +test('the handler sees the parsed payload and its decoded byte count', async () => { + const s = await startServer() + try { + const body = JSON.stringify({ resourceLogs: [{ resource: {} }] }) + const res = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body, + }) + assert.equal(res.status, 200) + assert.equal(s.seen.length, 1) + assert.equal(s.seen[0].signal, 'logs') + assert.deepEqual(s.seen[0].data, { resourceLogs: [{ resource: {} }] }) + assert.equal(s.seen[0].payloadBytes, Buffer.byteLength(body)) + } finally { + await s.close() + } +}) + +test('an empty body reads as an empty object rather than a parse error', async () => { + const s = await startServer() + try { + const res = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '', + }) + assert.equal(res.status, 200) + assert.deepEqual(s.seen[0].data, {}) + assert.equal(s.seen[0].payloadBytes, 0) + } finally { + await s.close() + } +}) + +test('a charset parameter on the content type is still application/json', async () => { + const s = await startServer() + try { + const res = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json; charset=utf-8' }, + body: '{}', + }) + assert.equal(res.status, 200) + } finally { + await s.close() + } +}) + +test('protobuf and a missing content type are both refused with 415', async () => { + const s = await startServer() + try { + const proto = await fetch(`${s.origin}/v1/traces`, { + method: 'POST', + headers: { 'Content-Type': 'application/x-protobuf' }, + body: 'not json', + }) + assert.equal(proto.status, 415) + const protoBody = /** @type {{ code: number, message: string }} */ (await proto.json()) + assert.equal(protoBody.code, 3) + assert.match(protoBody.message, /application\/x-protobuf/) + + // `fetch` always stamps a content type, so the header-less case needs a raw request. + const none = await postWithoutContentType(s.bound.port, '/v1/traces') + assert.equal(none.status, 415) + assert.match(none.body, /'none'/) + + assert.equal(s.seen.length, 0) + } finally { + await s.close() + } +}) + +test('gzip and deflate bodies are decoded before the handler sees them', async () => { + const s = await startServer() + try { + const payload = JSON.stringify({ resourceMetrics: [] }) + /** @type {[string, Buffer][]} */ + const cases = [ + ['gzip', zlib.gzipSync(payload)], + ['deflate', zlib.deflateSync(payload)], + ] + for (const [encoding, compressed] of cases) { + const res = await fetch(`${s.origin}/v1/metrics`, { + method: 'POST', + headers: { 'Content-Type': 'application/json', 'Content-Encoding': encoding }, + body: compressed, + }) + assert.equal(res.status, 200) + } + assert.equal(s.seen.length, 2) + for (const req of s.seen) { + assert.deepEqual(req.data, { resourceMetrics: [] }) + // The count is of decoded bytes, not of what came off the wire. + assert.equal(req.payloadBytes, Buffer.byteLength(payload)) + } + } finally { + await s.close() + } +}) + +test('an unknown content encoding is refused with 415', async () => { + const s = await startServer() + try { + const res = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json', 'Content-Encoding': 'br' }, + body: '{}', + }) + assert.equal(res.status, 415) + const body = /** @type {{ code: number, message: string }} */ (await res.json()) + assert.equal(body.code, 3) + assert.match(body.message, /Content-Encoding: br/) + assert.equal(s.seen.length, 0) + } finally { + await s.close() + } +}) + +test('a malformed JSON body is refused with 400 rather than crashing', async () => { + const s = await startServer() + try { + const res = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{ not json', + }) + assert.equal(res.status, 400) + assert.deepEqual(await res.json(), { code: 3, message: 'Invalid JSON' }) + assert.equal(s.seen.length, 0) + } finally { + await s.close() + } +}) + +test('an unknown path is 404 and a non-POST method is 405', async () => { + const s = await startServer() + try { + const missing = await fetch(`${s.origin}/v1/nope`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(missing.status, 404) + assert.deepEqual(await missing.json(), { code: 5, message: 'Not found' }) + + const wrongMethod = await fetch(`${s.origin}/v1/logs`, { method: 'PUT', body: '{}' }) + assert.equal(wrongMethod.status, 405) + assert.deepEqual(await wrongMethod.json(), { code: 12, message: 'Method not allowed' }) + } finally { + await s.close() + } +}) + +test('a handler failure becomes a 500 carrying its message', async () => { + const s = await startServer({ + onRequest() { + throw new Error('persist failed') + }, + }) + try { + const res = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(res.status, 500) + assert.deepEqual(await res.json(), { code: 13, message: 'persist failed' }) + } finally { + await s.close() + } +}) + +// @ref LLP 0256#control-route-on-listener [tests]: the reserved `/_hypaware/` +// prefix is a local control surface on the shared server too, short-circuited +// before OTLP routing, so the claude listener can host the session-ignore +// route with the identical shape the gateway proxy serves. +test('a registered control handler owns the reserved /_hypaware/ prefix, before OTLP routing', async () => { + /** @type {string[]} */ + const controlPaths = [] + const s = await startServer({ + onControlRequest(req, res, url) { + controlPaths.push(`${req.method} ${url.pathname}`) + req.resume() + res.writeHead(200, { 'content-type': 'application/json' }) + res.end(JSON.stringify({ ok: true })) + }, + }) + try { + // All three verbs the session-ignore route serves reach the handler, + // including GET, which the OTLP side would have refused with 405. + for (const method of ['GET', 'POST', 'DELETE']) { + const res = await fetch(`${s.origin}/_hypaware/ignore/session`, { + method, + ...(method === 'GET' ? {} : { headers: { 'content-type': 'application/json' }, body: '{}' }), + }) + assert.equal(res.status, 200, `${method} reaches the control handler`) + assert.deepEqual(await res.json(), { ok: true }) + } + assert.deepEqual(controlPaths, [ + 'GET /_hypaware/ignore/session', + 'POST /_hypaware/ignore/session', + 'DELETE /_hypaware/ignore/session', + ]) + assert.equal(s.seen.length, 0, 'a control request never reads as an OTLP export') + + // A look-alike path is NOT a control path, so it still routes as OTLP. + const lookAlike = await fetch(`${s.origin}/_hypawarefoo`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(lookAlike.status, 404) + assert.equal(controlPaths.length, 3, 'the look-alike never reached the control handler') + } finally { + await s.close() + } +}) + +test('without a control handler, control paths fall through as unknown OTLP routes', async () => { + const s = await startServer() + try { + const post = await fetch(`${s.origin}/_hypaware/ignore/session`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(post.status, 404) + assert.equal(s.seen.length, 0) + } finally { + await s.close() + } +}) + +test('a listener can serve a subset of the signals', async () => { + const s = await startServer({ signals: ['logs', 'metrics'] }) + try { + const logs = await fetch(`${s.origin}/v1/logs`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(logs.status, 200) + + const traces = await fetch(`${s.origin}/v1/traces`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }) + assert.equal(traces.status, 404) + assert.equal(s.seen.length, 1) + } finally { + await s.close() + } +}) diff --git a/test/core/purge-command.test.js b/test/core/purge-command.test.js index 5a52921a..e93f96c8 100644 --- a/test/core/purge-command.test.js +++ b/test/core/purge-command.test.js @@ -13,6 +13,7 @@ import { resolveIcebergDir } from '../../src/core/cache/storage.js' import { readRowsFromTable, scanRowsFromTable } from '../../src/core/cache/iceberg/store.js' import { runPurge } from '../../src/core/commands/purge.js' import { scopeGovernance, scopeGoverns } from '../../src/core/usage-policy/matcher.js' +import { claudeBodySpoolDir } from '../../hypaware-core/plugins-workspace/claude/src/telemetry/spool.js' /** * @import { ColumnSpec } from '../../hypaware-plugin-kernel-types.js' @@ -354,6 +355,57 @@ test('runPurge --all --yes deletes everything and reports counts', async () => { } }) +// The spool holds raw request and response bodies that have not been projected +// yet. Leaving them behind would let the next received batch write rows the +// user just deleted, so every purge empties it - a targeted purge included, +// because a spooled body carries no cwd for a target to match against. +// @ref LLP 0253#purge-and-detach-sweep [tests]: `hyp purge` removes the spool +// directory's contents +test('runPurge empties the capture spool and reports what it removed', async () => { + const cacheRoot = await makeTmpDir('cli-spool') + const hypHome = await makeTmpDir('cli-spool-home') + try { + await seed(cacheRoot) + const spoolDir = claudeBodySpoolDir(hypHome) + await fs.mkdir(spoolDir, { recursive: true }) + await fs.writeFile(path.join(spoolDir, 'req-1.json'), '{"messages":["a raw prompt"]}') + await fs.writeFile(path.join(spoolDir, 'resp-1.json'), '{"content":["a raw reply"]}') + + const { ctx, stdout } = makeCtx({ cacheRoot, hypHome }) + const code = await runPurge(['--all', '--yes', '--json'], ctx) + assert.equal(code, 0) + assert.equal(JSON.parse(stdout.text).spoolFilesRemoved, 2) + assert.deepEqual(await fs.readdir(spoolDir), [], 'the directory survives, its contents do not') + } finally { + await fs.rm(cacheRoot, { recursive: true, force: true }) + await fs.rm(hypHome, { recursive: true, force: true }) + } +}) + +test('runPurge sweeps the spool for a targeted purge too, and says nothing when it is empty', async () => { + const cacheRoot = await makeTmpDir('cli-spool-target') + const hypHome = await makeTmpDir('cli-spool-target-home') + try { + await seed(cacheRoot) + const spoolDir = claudeBodySpoolDir(hypHome) + await fs.mkdir(spoolDir, { recursive: true }) + await fs.writeFile(path.join(spoolDir, 'req-1.json'), '{"messages":["a raw prompt"]}') + + const { ctx, stdout } = makeCtx({ cacheRoot, hypHome }) + assert.equal(await runPurge([REPO_B, '--yes'], ctx), 0) + assert.match(stdout.text, /emptied the capture spool: 1 raw body file deleted/) + assert.deepEqual(await fs.readdir(spoolDir), []) + + // A second purge finds nothing and stays quiet about it. + const second = makeCtx({ cacheRoot, hypHome }) + assert.equal(await runPurge([REPO_B, '--yes'], second.ctx), 0) + assert.doesNotMatch(second.stdout.text, /capture spool/) + } finally { + await fs.rm(cacheRoot, { recursive: true, force: true }) + await fs.rm(hypHome, { recursive: true, force: true }) + } +}) + test('runPurge subtree warns about resurrection when the dir still resolves full', async () => { const cacheRoot = await makeTmpDir('cli-warn') const hypHome = await makeTmpDir('cli-warn-home') diff --git a/test/plugins/ai-gateway-control-route.test.js b/test/core/session-ignore-control-route.test.js similarity index 98% rename from test/plugins/ai-gateway-control-route.test.js rename to test/core/session-ignore-control-route.test.js index db05d531..4c7b7d4a 100644 --- a/test/plugins/ai-gateway-control-route.test.js +++ b/test/core/session-ignore-control-route.test.js @@ -4,8 +4,7 @@ import assert from 'node:assert/strict' import http from 'node:http' import test from 'node:test' -import { createControlHandler } from '../../hypaware-core/plugins-workspace/ai-gateway/src/control.js' -import { isControlPath } from '../../hypaware-core/plugins-workspace/ai-gateway/src/proxy.js' +import { createControlHandler, isControlPath } from '../../src/core/control/session_ignore.js' /** * @import { IncomingMessage, ServerResponse } from 'node:http' diff --git a/test/core/status-attach-mode.test.js b/test/core/status-attach-mode.test.js new file mode 100644 index 00000000..eaac8b11 --- /dev/null +++ b/test/core/status-attach-mode.test.js @@ -0,0 +1,145 @@ +// @ts-check + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { collectHypAwareStatus } from '../../src/core/daemon/status.js' +import { renderStatusText } from '../../src/core/commands/status.js' +import { defaultConfigPath } from '../../src/core/config/schema.js' + +/** + * The attach mode on the text surface. `--json` has carried + * `client_attach[].mode` since the marker grew one; a machine the LLP 0262 + * migration just moved from `proxy` to `otel` must be readable off the plain + * `hyp status` too, or the migration's outcome is invisible on the surface a + * human actually checks. Markers that predate modes keep the bare word. + * + * @ref LLP 0262#migration [tests]: hyp status reflects the new attach mode after the migration + */ + +async function makeHome() { + const hypHome = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-status-attach-mode-')) + await fs.mkdir(path.join(hypHome, 'hypaware'), { recursive: true }) + await fs.writeFile(defaultConfigPath(hypHome), JSON.stringify({ version: 2, plugins: [] }) + '\n') + return hypHome +} + +/** @param {string} hypHome */ +function env(hypHome) { + return { ...process.env, HYP_HOME: hypHome, HYP_CONFIG: '' } +} + +function makeBuf() { + let value = '' + return { + write(/** @type {string} */ chunk) { + value += String(chunk) + return true + }, + text() { + return value + }, + } +} + +/** + * A report whose client list these renderer tests own outright, the same + * move status-client-error.test.js makes: catalog rows for the same names + * would otherwise shadow the rows under test. + * + * @param {string} hypHome + * @param {Array>} clients + */ +async function reportWithClients(hypHome, clients) { + const report = await collectHypAwareStatus({ env: env(hypHome) }) + report.clients = /** @type {any} */ (clients) + return report +} + +test('an attached client renders its marker mode on the text surface', async () => { + const hypHome = await makeHome() + const report = await reportWithClients(hypHome, [ + { name: 'claude', plugin: '@hypaware/claude', configured: true, attached: true, mode: 'otel' }, + { name: 'codex', plugin: '@hypaware/codex', configured: true, attached: true, mode: 'base_url' }, + ]) + + const stdout = makeBuf() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: '/tmp/cache', stdout }) + const text = stdout.text() + + assert.match(text, /- claude {2}\[configured, attached \(otel\)\]/) + assert.match(text, /- codex {2}\[configured, attached \(base_url\)\]/) +}) + +test('a mode-less marker and a detached client keep the bare words', async () => { + const hypHome = await makeHome() + const report = await reportWithClients(hypHome, [ + { name: 'claude', plugin: '@hypaware/claude', configured: true, attached: true }, + { name: 'codex', plugin: '@hypaware/codex', configured: true, attached: false }, + ]) + + const stdout = makeBuf() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: '/tmp/cache', stdout }) + const text = stdout.text() + + assert.match(text, /- claude {2}\[configured, attached\]/) + assert.match(text, /- codex {2}\[configured, not attached\]/) +}) + +test('a probe-less client says attach n/a whatever mode its marker claims', async () => { + const hypHome = await makeHome() + const report = await reportWithClients(hypHome, [ + { + name: 'hermes', + plugin: '@hypaware/hermes', + configured: true, + attachable: false, + attached: true, + mode: 'otel', + }, + ]) + + const stdout = makeBuf() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: '/tmp/cache', stdout }) + + assert.match(stdout.text(), /- hermes {2}\[configured, attach n\/a\]/) +}) + +/** + * The mode is read back out of the client's own settings file, which a hand + * edit reaches, so it is a captured label rather than an in-process constant. + * + * @ref LLP 0225#one-vocabulary [tests]: a label lifted off disk cannot drive the terminal it is printed to + */ +test('a marker mode carrying terminal control bytes is stripped before it is printed', async () => { + const hypHome = await makeHome() + const report = await reportWithClients(hypHome, [ + { + name: 'claude', + plugin: '@hypaware/claude', + configured: true, + attached: true, + mode: 'ot\u001b[2Kel\n', + }, + { + name: 'codex', + plugin: '@hypaware/codex', + configured: true, + attached: true, + mode: '\u200b\u200b', + }, + ]) + + const stdout = makeBuf() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: '/tmp/cache', stdout }) + const text = stdout.text() + + assert.ok(!text.includes('\u001b'), 'no escape byte reaches the terminal') + assert.match(text, /- claude {2}\[configured, attached \(ot\[2Kel\)\]/) + // A mode that sanitizes away to nothing leaves the bare word rather than an + // empty parenthesis. + assert.match(text, /- codex {2}\[configured, attached\]/) +}) diff --git a/test/core/status-capture-health.test.js b/test/core/status-capture-health.test.js new file mode 100644 index 00000000..14c58050 --- /dev/null +++ b/test/core/status-capture-health.test.js @@ -0,0 +1,575 @@ +// @ts-check + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { + CAPTURE_GAP_ERROR_MS, + CAPTURE_GAP_WARNING_MS, + assessCaptureHealth, + collectHypAwareStatus, + formatGapDuration, + probeClientActivityFromDescriptor, + writeStatusFile, +} from '../../src/core/daemon/status.js' +import { writePidFile } from '../../src/core/daemon/pid.js' +import { renderStatusJson, renderStatusText } from '../../src/core/commands/status.js' +import { defaultConfigPath } from '../../src/core/config/schema.js' + +/** @import { CollectStatusOptions } from '../../src/core/daemon/types.js' */ + +// The capture-health line (LLP 0257 S17, the RFC 0262 open-question-1 duty): +// on the otel path a broken exporter, a stale endpoint, and a down daemon all +// fail into the same silence, so `hyp status` holds the client's own +// transcript trail against the last event the listener recorded and gets +// loud when they diverge. +// @ref LLP 0257#status-and-health [tests]: + +const MIN = 60_000 +const HOUR = 3_600_000 + +async function makeHome() { + const hypHome = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-status-capture-')) + const stateRoot = path.join(hypHome, 'hypaware') + await fs.mkdir(path.join(stateRoot, 'run'), { recursive: true }) + await fs.writeFile(defaultConfigPath(hypHome), JSON.stringify({ + version: 2, + plugins: [ + { + name: '@hypaware/ai-gateway', + config: { + listen: '127.0.0.1:8787', + upstreams: [ + { name: 'anthropic', base_url: 'https://api.anthropic.com', path_prefix: '/' }, + ], + }, + }, + { name: '@hypaware/claude', config: { proxy: '@hypaware/ai-gateway' } }, + ], + }) + '\n') + return { hypHome, stateRoot } +} + +/** + * A fake $HOME whose `.claude/settings.json` carries an attach marker, and + * whose `.claude/projects` tree holds one transcript with a chosen mtime. + * + * @param {{ mode?: string, attachedAt?: string, transcriptMtime?: Date }} [opts] + */ +async function makeClientHome(opts = {}) { + const home = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-status-capture-home-')) + await fs.mkdir(path.join(home, '.claude'), { recursive: true }) + if (opts.mode !== undefined) { + await fs.writeFile(path.join(home, '.claude', 'settings.json'), JSON.stringify({ + _hypaware: { + attached_at: opts.attachedAt ?? new Date(Date.now() - 24 * HOUR).toISOString(), + version: '2.0.0', + port: 8787, + mode: opts.mode, + managed: { env: {}, hooks: [] }, + }, + env: {}, + }) + '\n') + } + if (opts.transcriptMtime) { + const dir = path.join(home, '.claude', 'projects', '-Users-t-proj') + await fs.mkdir(dir, { recursive: true }) + const file = path.join(dir, 'aaaa-session.jsonl') + await fs.writeFile(file, '{}\n') + await fs.utimes(file, opts.transcriptMtime, opts.transcriptMtime) + } + return home +} + +/** + * @param {string} stateRoot + * @param {string | null} lastEventAt + */ +function writeDaemonStatus(stateRoot, lastEventAt) { + writeStatusFile(stateRoot, /** @type {any} */ ({ + state: 'healthy', + sources: [ + { + name: 'ai-gateway', + plugin: '@hypaware/ai-gateway', + state: 'started', + details: { host: '127.0.0.1', port: 8787 }, + }, + { + name: 'claude-telemetry', + plugin: '@hypaware/claude', + state: 'started', + details: { listen_host: '127.0.0.1', listen_port: 4319, last_event_at: lastEventAt }, + }, + ], + sinks: [], + })) +} + +/** + * @param {string} hypHome + * @param {string} homeDir + * @returns {CollectStatusOptions} + */ +function collectOpts(hypHome, homeDir) { + return { + env: { ...process.env, HYP_HOME: hypHome, HYP_CONFIG: '' }, + homeDir, + platform: 'darwin', + isLaunchAgentInstalled: () => false, + } +} + +/** @returns {{ write(chunk: string): void, text(): string }} */ +function buffer() { + /** @type {string[]} */ + const chunks = [] + return { write: (chunk) => { chunks.push(chunk) }, text: () => chunks.join('') } +} + +/** @param {string} hypHome @param {string} homeDir */ +async function cleanup(hypHome, homeDir) { + await fs.rm(hypHome, { recursive: true, force: true }) + await fs.rm(homeDir, { recursive: true, force: true }) +} + +/* ---------- assessCaptureHealth: the threshold contract ---------- */ + +test('capture in lockstep is ok, and a transcript slightly ahead stays under the threshold', () => { + const now = Date.now() + const ok = assessCaptureHealth({ + lastEventAt: new Date(now - 2 * MIN).toISOString(), + lastTranscriptActivityAt: new Date(now - 1 * MIN).toISOString(), + attachedAt: new Date(now - 5 * HOUR).toISOString(), + }) + assert.equal(ok.state, 'ok') + assert.equal(ok.gapMs, 1 * MIN) +}) + +test('a transcript past the warning threshold is a warning gap, past the error threshold an error', () => { + const now = Date.now() + const warn = assessCaptureHealth({ + lastEventAt: new Date(now - 30 * MIN).toISOString(), + lastTranscriptActivityAt: new Date(now).toISOString(), + attachedAt: new Date(now - 5 * HOUR).toISOString(), + }) + assert.equal(warn.state, 'gap') + assert.equal(warn.severity, 'warning') + assert.equal(warn.gapMs, 30 * MIN) + + const error = assessCaptureHealth({ + lastEventAt: new Date(now - 5 * HOUR).toISOString(), + lastTranscriptActivityAt: new Date(now).toISOString(), + attachedAt: new Date(now - 6 * HOUR).toISOString(), + }) + assert.equal(error.state, 'gap') + assert.equal(error.severity, 'error') +}) + +test('the boundary values sit exactly on the documented thresholds', () => { + const base = Date.parse('2026-08-17T12:00:00.000Z') + const at = (/** @type {number} */ ms) => new Date(ms).toISOString() + const attachedAt = at(base - 24 * HOUR) + const onWarn = assessCaptureHealth({ + lastEventAt: at(base), + lastTranscriptActivityAt: at(base + CAPTURE_GAP_WARNING_MS), + attachedAt, + }) + assert.equal(onWarn.state, 'ok') + const pastWarn = assessCaptureHealth({ + lastEventAt: at(base), + lastTranscriptActivityAt: at(base + CAPTURE_GAP_WARNING_MS + 1), + attachedAt, + }) + assert.deepEqual([pastWarn.state, pastWarn.severity], ['gap', 'warning']) + const onError = assessCaptureHealth({ + lastEventAt: at(base), + lastTranscriptActivityAt: at(base + CAPTURE_GAP_ERROR_MS), + attachedAt, + }) + assert.deepEqual([onError.state, onError.severity], ['gap', 'warning']) + const pastError = assessCaptureHealth({ + lastEventAt: at(base), + lastTranscriptActivityAt: at(base + CAPTURE_GAP_ERROR_MS + 1), + attachedAt, + }) + assert.deepEqual([pastError.state, pastError.severity], ['gap', 'error']) +}) + +test('with no events the attach timestamp is the baseline, and pre-attach activity proves nothing', () => { + const now = Date.now() + // Months of transcripts from before the attach: the usual shape right + // after a proxy-to-otel migration. Not a gap. + const preAttach = assessCaptureHealth({ + lastEventAt: null, + lastTranscriptActivityAt: new Date(now - 3 * 24 * HOUR).toISOString(), + attachedAt: new Date(now - 1 * HOUR).toISOString(), + }) + assert.deepEqual([preAttach.state, preAttach.gapMs], ['ok', 0]) + // Activity after the attach with still no events is the broken-path shape. + const broken = assessCaptureHealth({ + lastEventAt: null, + lastTranscriptActivityAt: new Date(now - 1 * MIN).toISOString(), + attachedAt: new Date(now - 1 * HOUR).toISOString(), + }) + assert.deepEqual([broken.state, broken.severity], ['gap', 'warning']) +}) + +test('missing halves never fabricate a gap', () => { + const now = new Date().toISOString() + assert.equal(assessCaptureHealth({ lastEventAt: now, lastTranscriptActivityAt: null, attachedAt: now }).state, 'ok') + assert.equal(assessCaptureHealth({ lastEventAt: null, lastTranscriptActivityAt: now, attachedAt: null }).state, 'ok') + assert.equal( + assessCaptureHealth({ lastEventAt: 'not a date', lastTranscriptActivityAt: now, attachedAt: 'nope' }).state, + 'ok' + ) +}) + +test('formatGapDuration is coarse: minutes, then hours, then days', () => { + assert.equal(formatGapDuration(20 * MIN), '20m') + assert.equal(formatGapDuration(90 * MIN), '1h') + assert.equal(formatGapDuration(30 * HOUR), '30h') + assert.equal(formatGapDuration(3 * 24 * HOUR), '3d') +}) + +/* ---------- probeClientActivityFromDescriptor ---------- */ + +test('the activity probe reports the newest matching mtime, filtered by suffix', async () => { + const home = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-activity-probe-')) + try { + const projects = path.join(home, '.claude', 'projects') + const older = path.join(projects, 'proj-a', 'old.jsonl') + const newest = path.join(projects, 'proj-b', 'sess', 'subagents', 'agent-1.jsonl') + const decoy = path.join(projects, 'proj-b', 'newer-but-wrong-suffix.txt') + for (const file of [older, newest, decoy]) { + await fs.mkdir(path.dirname(file), { recursive: true }) + await fs.writeFile(file, '{}\n') + } + const t0 = new Date('2026-08-17T10:00:00.000Z') + const t1 = new Date('2026-08-17T11:00:00.000Z') + const t2 = new Date('2026-08-17T12:00:00.000Z') + await fs.utimes(older, t0, t0) + await fs.utimes(newest, t1, t1) + await fs.utimes(decoy, t2, t2) + + const descriptor = /** @type {any} */ ({ + plugin: '@hypaware/claude', + name: 'claude', + skillDir: '.claude/skills', + activityProbe: { dir: '.claude/projects', file_suffix: '.jsonl' }, + }) + const seen = await probeClientActivityFromDescriptor({ descriptor, homeDir: home, env: {} }) + assert.equal(seen, t1.toISOString()) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +test('a missing tree, a missing probe, and an escaping dir all read as no claim', async () => { + const home = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-activity-probe-')) + try { + const base = /** @type {any} */ ({ plugin: '@hypaware/claude', name: 'claude', skillDir: '.claude/skills' }) + assert.equal( + await probeClientActivityFromDescriptor({ + descriptor: { ...base, activityProbe: { dir: '.claude/projects' } }, + homeDir: home, + env: {}, + }), + undefined + ) + assert.equal(await probeClientActivityFromDescriptor({ descriptor: base, homeDir: home, env: {} }), undefined) + assert.equal( + await probeClientActivityFromDescriptor({ + descriptor: { ...base, activityProbe: { dir: '../outside' } }, + homeDir: home, + env: {}, + }), + undefined + ) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } +}) + +/* ---------- collect + render ---------- */ + +test('an otel-attached client in lockstep renders the line, healthy, in text and json', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + const home = await makeClientHome({ + mode: 'otel', + attachedAt: new Date(now - 6 * HOUR).toISOString(), + transcriptMtime: new Date(now - 1 * MIN), + }) + try { + writeDaemonStatus(stateRoot, new Date(now - 2 * MIN).toISOString()) + + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.captureHealth.length, 1) + const health = report.captureHealth[0] + assert.equal(health.client, 'claude') + assert.equal(health.source, 'claude-telemetry') + assert.equal(health.state, 'ok') + assert.equal(report.diagnostics.some((d) => d.kind === 'capture_gap'), false) + assert.equal(report.overall, 'healthy') + + const stdout = buffer() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache'), stdout }) + const text = stdout.text() + assert.match(text, /capture health:/) + assert.match(text, /- claude {2}last event 2m ago, last transcript activity 1m ago\n/) + assert.doesNotMatch(text, /\[capture gap\]/) + + const json = renderStatusJson({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache') }) + assert.equal(json.capture_health.length, 1) + assert.equal(json.capture_health[0].client, 'claude') + assert.equal(json.capture_health[0].state, 'ok') + assert.equal(typeof json.capture_health[0].last_event_at, 'string') + assert.equal(typeof json.capture_health[0].last_transcript_activity_at, 'string') + // The attach marker's mode rides the client_attach entry (LLP 0258). + const claude = json.client_attach.find((/** @type {any} */ c) => c.name === 'claude') + assert.equal(claude?.mode, 'otel') + } finally { + await cleanup(hypHome, home) + } +}) + +test('transcripts running hours past the last event degrade overall through an error diagnostic', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + const home = await makeClientHome({ + mode: 'otel', + attachedAt: new Date(now - 24 * HOUR).toISOString(), + transcriptMtime: new Date(now - 1 * MIN), + }) + try { + writeDaemonStatus(stateRoot, new Date(now - 5 * HOUR).toISOString()) + + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.captureHealth[0]?.state, 'gap') + const diag = report.diagnostics.find((d) => d.kind === 'capture_gap') + assert.ok(diag, JSON.stringify(report.diagnostics, null, 2)) + assert.equal(diag.severity, 'error') + assert.match(diag.message, /not being captured/) + assert.ok(diag.repair.some((r) => r.includes('hyp daemon restart'))) + assert.ok(diag.repair.some((r) => r.includes('hyp attach --client claude'))) + assert.equal(report.overall, 'degraded') + + const stdout = buffer() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache'), stdout }) + assert.match(stdout.text(), /- claude {2}last event 5h ago, last transcript activity 1m ago {2}\[capture gap\]\n/) + } finally { + await cleanup(hypHome, home) + } +}) + +test('a moderate gap warns without degrading overall', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + const home = await makeClientHome({ + mode: 'otel', + attachedAt: new Date(now - 24 * HOUR).toISOString(), + transcriptMtime: new Date(now - 1 * MIN), + }) + try { + writeDaemonStatus(stateRoot, new Date(now - 40 * MIN).toISOString()) + + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + const diag = report.diagnostics.find((d) => d.kind === 'capture_gap') + assert.equal(diag?.severity, 'warning') + assert.equal(report.overall, 'healthy') + } finally { + await cleanup(hypHome, home) + } +}) + +test('no marker and a non-otel marker both keep the surface silent', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + for (const mode of [/** @type {string | undefined} */ (undefined), 'proxy']) { + const home = await makeClientHome({ mode, transcriptMtime: new Date(now - 1 * MIN) }) + try { + writeDaemonStatus(stateRoot, new Date(now - 5 * HOUR).toISOString()) + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.deepEqual(report.captureHealth, [], `mode=${String(mode)}`) + assert.equal(report.diagnostics.some((d) => d.kind === 'capture_gap'), false) + + const stdout = buffer() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache'), stdout }) + assert.doesNotMatch(stdout.text(), /capture health/) + + const json = renderStatusJson({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache') }) + assert.deepEqual(json.capture_health, []) + } finally { + await fs.rm(home, { recursive: true, force: true }) + } + } + await fs.rm(hypHome, { recursive: true, force: true }) +}) + +test('a daemon that never ran still yields the line, measured from the attach', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + const home = await makeClientHome({ + mode: 'otel', + attachedAt: new Date(now - 2 * HOUR).toISOString(), + transcriptMtime: new Date(now - 1 * MIN), + }) + try { + // No status.json at all: attach ran, the daemon never did. The listener + // recorded nothing, and the transcripts kept moving. + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.captureHealth.length, 1) + const health = report.captureHealth[0] + assert.equal(health.source, null) + assert.equal(health.lastEventAt, null) + assert.equal(health.state, 'gap') + const diag = report.diagnostics.find((d) => d.kind === 'capture_gap') + assert.match(diag?.message ?? '', /no telemetry has arrived/) + + const stdout = buffer() + renderStatusText({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache'), stdout }) + assert.match(stdout.text(), /- claude {2}no events yet, last transcript activity 1m ago {2}\[capture gap\]\n/) + } finally { + await cleanup(hypHome, home) + } +}) + +/* ---------- the restart baseline ---------- */ + +// `state.lastEventAt` lives only in the listener's process, so every daemon +// restart republishes `last_event_at: null` however long capture has been +// healthy. With the attach timestamp as the only fallback baseline, a machine +// attached a month ago and used an hour ago reported a month-long gap - an +// `error`, degrading `overall` - the moment someone ran `hyp daemon restart`, +// which is itself the first repair `capture_gap` prints. The running +// listener's own start is the third baseline that closes that loop. +// @ref LLP 0257#status-and-health [tests]: the gap is measured from a moment capture was actually supposed to be running + +test('a listener that just started cannot be blamed for activity older than it', () => { + const now = Date.now() + const restarted = assessCaptureHealth({ + lastEventAt: null, + lastTranscriptActivityAt: new Date(now - 1 * HOUR).toISOString(), + attachedAt: new Date(now - 30 * 24 * HOUR).toISOString(), + listenerStartedAt: new Date(now - 1 * MIN).toISOString(), + }) + assert.deepEqual([restarted.state, restarted.gapMs], ['ok', 0]) +}) + +test('a listener up long enough to have seen something still reports the gap', () => { + const now = Date.now() + const real = assessCaptureHealth({ + lastEventAt: null, + lastTranscriptActivityAt: new Date(now - 1 * MIN).toISOString(), + attachedAt: new Date(now - 30 * 24 * HOUR).toISOString(), + listenerStartedAt: new Date(now - 5 * HOUR).toISOString(), + }) + assert.deepEqual([real.state, real.severity], ['gap', 'error']) +}) + +test('an event newer than the listener start still wins the baseline', () => { + const now = Date.now() + const verdict = assessCaptureHealth({ + lastEventAt: new Date(now - 1 * MIN).toISOString(), + lastTranscriptActivityAt: new Date(now).toISOString(), + attachedAt: new Date(now - 30 * 24 * HOUR).toISOString(), + listenerStartedAt: new Date(now - 5 * HOUR).toISOString(), + }) + assert.equal(verdict.state, 'ok') +}) + +test('a routine daemon restart does not degrade a healthy install', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + const home = await makeClientHome({ + mode: 'otel', + attachedAt: new Date(now - 30 * 24 * HOUR).toISOString(), + transcriptMtime: new Date(now - 1 * HOUR), + }) + try { + // A live daemon whose listener came up a minute ago and has not been + // POSTed to yet, because no Claude Code session has started since. + writeStatusFile(stateRoot, /** @type {any} */ ({ + state: 'healthy', + sources: [ + { + name: 'ai-gateway', + plugin: '@hypaware/ai-gateway', + state: 'started', + details: { host: '127.0.0.1', port: 8787 }, + }, + { + name: 'claude-telemetry', + plugin: '@hypaware/claude', + state: 'started', + details: { + listen_host: '127.0.0.1', + listen_port: 4319, + last_event_at: null, + listener_started_at: new Date(now - 1 * MIN).toISOString(), + }, + }, + ], + sinks: [], + })) + writePidFile(stateRoot, /** @type {any} */ ({ + pid: process.pid, + runId: 'test-run', + mode: 'foreground', + })) + + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.captureHealth[0]?.state, 'ok') + assert.equal(report.diagnostics.some((d) => d.kind === 'capture_gap'), false) + assert.equal(report.overall, 'healthy') + assert.equal(typeof report.captureHealth[0]?.listenerStartedAt, 'string') + + const json = renderStatusJson({ report, clientNames: [], datasets: [], cacheRoot: path.join(stateRoot, 'cache') }) + assert.equal(typeof json.capture_health[0].listener_started_at, 'string') + } finally { + await cleanup(hypHome, home) + } +}) + +test('a dead daemon makes no restart excuse: its listener start bounds nothing now', async () => { + const { hypHome, stateRoot } = await makeHome() + const now = Date.now() + const home = await makeClientHome({ + mode: 'otel', + attachedAt: new Date(now - 30 * 24 * HOUR).toISOString(), + transcriptMtime: new Date(now - 1 * HOUR), + }) + try { + // The same snapshot as above, but no pid file: the daemon that wrote it is + // gone. "It only just started" stopped being true when the process ended, + // and a daemon-down gap is exactly what this line exists to surface. + writeStatusFile(stateRoot, /** @type {any} */ ({ + state: 'healthy', + sources: [ + { + name: 'claude-telemetry', + plugin: '@hypaware/claude', + state: 'started', + details: { + listen_host: '127.0.0.1', + listen_port: 4319, + last_event_at: null, + listener_started_at: new Date(now - 1 * MIN).toISOString(), + }, + }, + ], + sinks: [], + })) + + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.captureHealth[0]?.state, 'gap') + assert.equal(report.captureHealth[0]?.listenerStartedAt, null) + assert.equal(report.diagnostics.find((d) => d.kind === 'capture_gap')?.severity, 'error') + } finally { + await cleanup(hypHome, home) + } +}) diff --git a/test/core/status-telemetry-endpoint-drift.test.js b/test/core/status-telemetry-endpoint-drift.test.js new file mode 100644 index 00000000..fb35ad7e --- /dev/null +++ b/test/core/status-telemetry-endpoint-drift.test.js @@ -0,0 +1,206 @@ +// @ts-check + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { collectHypAwareStatus, writeStatusFile } from '../../src/core/daemon/status.js' +import { writePidFile } from '../../src/core/daemon/pid.js' +import { defaultConfigPath } from '../../src/core/config/schema.js' + +/** @import { CollectStatusOptions } from '../../src/core/daemon/types.js' */ + +// An `otel` attach writes ONE endpoint into the client's settings and nothing +// ever rewrites it. The listener, meanwhile, falls back to an ephemeral port +// when its default is taken (LLP 0114 §ephemeral-fallback), and an attach that +// ran with no live daemon could only write the default in the first place. The +// two drifting apart is silent capture loss with every other status line +// healthy - and the client keeps POSTing prompts and responses at whatever +// process holds the port it was told about. `client_telemetry_stale` is that +// comparison, made from data already on disk. +// +// @ref LLP 0114#fallback-is-visible [tests]: a listener on its ephemeral fallback is visible in status, here through the client left pointing at the port it vacated +// @ref LLP 0086#status-drift-diagnostic [tests]: warn and name the repair, against the port this attach mode actually writes + +const HOUR = 3_600_000 + +async function makeHome() { + const hypHome = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-telemetry-drift-')) + const stateRoot = path.join(hypHome, 'hypaware') + await fs.mkdir(path.join(stateRoot, 'run'), { recursive: true }) + await fs.writeFile(defaultConfigPath(hypHome), JSON.stringify({ + version: 2, + plugins: [ + { + name: '@hypaware/ai-gateway', + config: { + listen: '127.0.0.1:8787', + upstreams: [ + { name: 'anthropic', base_url: 'https://api.anthropic.com', path_prefix: '/' }, + ], + }, + }, + { name: '@hypaware/claude', config: { proxy: '@hypaware/ai-gateway' } }, + ], + }) + '\n') + return { hypHome, stateRoot } +} + +/** + * A fake $HOME carrying an `otel` attach marker whose managed env points the + * exporter at `endpoint`, plus one transcript so the capture-health block has + * both halves to work with. + * + * @param {{ endpoint?: string, transcriptMtime?: Date }} [opts] + */ +async function makeClientHome(opts = {}) { + const home = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-telemetry-drift-home-')) + await fs.mkdir(path.join(home, '.claude'), { recursive: true }) + const env = opts.endpoint === undefined + ? {} + : { OTEL_EXPORTER_OTLP_ENDPOINT: opts.endpoint } + await fs.writeFile(path.join(home, '.claude', 'settings.json'), JSON.stringify({ + _hypaware: { + attached_at: new Date(Date.now() - 6 * HOUR).toISOString(), + version: '2.0.0', + port: 8787, + mode: 'otel', + managed: { env, hooks: [] }, + }, + env, + }) + '\n') + const mtime = opts.transcriptMtime ?? new Date(Date.now() - 60_000) + const dir = path.join(home, '.claude', 'projects', '-Users-t-proj') + await fs.mkdir(dir, { recursive: true }) + const file = path.join(dir, 'aaaa-session.jsonl') + await fs.writeFile(file, '{}\n') + await fs.utimes(file, mtime, mtime) + return home +} + +/** + * @param {string} stateRoot + * @param {{ listenPort: number, running: boolean }} args + */ +function writeDaemon(stateRoot, { listenPort, running }) { + writeStatusFile(stateRoot, /** @type {any} */ ({ + state: 'healthy', + sources: [ + { + name: 'ai-gateway', + plugin: '@hypaware/ai-gateway', + state: 'started', + details: { host: '127.0.0.1', port: 8787 }, + }, + { + name: 'claude-telemetry', + plugin: '@hypaware/claude', + state: 'started', + details: { + listen_host: '127.0.0.1', + listen_port: listenPort, + last_event_at: new Date(Date.now() - 120_000).toISOString(), + }, + }, + ], + sinks: [], + })) + if (running) { + writePidFile(stateRoot, /** @type {any} */ ({ + pid: process.pid, + runId: 'test-run', + mode: 'foreground', + })) + } +} + +/** + * @param {string} hypHome + * @param {string} homeDir + * @returns {CollectStatusOptions} + */ +function collectOpts(hypHome, homeDir) { + return { + env: { ...process.env, HYP_HOME: hypHome, HYP_CONFIG: '' }, + homeDir, + platform: 'darwin', + isLaunchAgentInstalled: () => false, + } +} + +/** @param {string} hypHome @param {string} homeDir */ +async function cleanup(hypHome, homeDir) { + await fs.rm(hypHome, { recursive: true, force: true }) + await fs.rm(homeDir, { recursive: true, force: true }) +} + +test('a client exporting to the port the listener vacated is a warning naming both ports', async () => { + const { hypHome, stateRoot } = await makeHome() + const home = await makeClientHome({ endpoint: 'http://127.0.0.1:4319' }) + try { + // The default was taken at boot, so the listener fell back to an + // ephemeral bind - and 4319 is now held by someone else entirely. + writeDaemon(stateRoot, { listenPort: 54321, running: true }) + + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + const stale = report.diagnostics.find((d) => d.kind === 'client_telemetry_stale') + assert.ok(stale, 'expected a client_telemetry_stale diagnostic') + assert.equal(stale.severity, 'warning') + assert.match(stale.message, /port 4319/) + assert.match(stale.message, /port 54321/) + assert.ok(stale.repair.includes('hyp attach --client claude')) + // Non-degrading, like every other attach-drift warning. + assert.equal(report.overall, 'healthy') + + const claude = report.clients.find((c) => c.name === 'claude') + assert.equal(claude?.telemetryPort, 4319) + } finally { + await cleanup(hypHome, home) + } +}) + +test('agreement between the marker endpoint and the live bind raises nothing', async () => { + const { hypHome, stateRoot } = await makeHome() + const home = await makeClientHome({ endpoint: 'http://127.0.0.1:4319' }) + try { + writeDaemon(stateRoot, { listenPort: 4319, running: true }) + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.diagnostics.some((d) => d.kind === 'client_telemetry_stale'), false) + assert.equal(report.clients.find((c) => c.name === 'claude')?.telemetryPort, 4319) + } finally { + await cleanup(hypHome, home) + } +}) + +test('a dead daemon makes no drift claim: its snapshot cannot say where anything is bound now', async () => { + const { hypHome, stateRoot } = await makeHome() + const home = await makeClientHome({ endpoint: 'http://127.0.0.1:4319' }) + try { + writeDaemon(stateRoot, { listenPort: 54321, running: false }) + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal(report.diagnostics.some((d) => d.kind === 'client_telemetry_stale'), false) + } finally { + await cleanup(hypHome, home) + } +}) + +test('a marker with no telemetry endpoint, or a nonsense one, reads as no claim', async () => { + for (const endpoint of [undefined, 'not a url', 'http://127.0.0.1/', 'http://127.0.0.1:0']) { + const { hypHome, stateRoot } = await makeHome() + const home = await makeClientHome({ endpoint }) + try { + writeDaemon(stateRoot, { listenPort: 54321, running: true }) + const report = await collectHypAwareStatus(collectOpts(hypHome, home)) + assert.equal( + report.diagnostics.some((d) => d.kind === 'client_telemetry_stale'), + false, + `expected no drift claim for endpoint ${String(endpoint)}` + ) + assert.equal(report.clients.find((c) => c.name === 'claude')?.telemetryPort, undefined) + } finally { + await cleanup(hypHome, home) + } + } +}) diff --git a/test/plugins/ai-gateway-exchange-writer.test.js b/test/plugins/ai-gateway-exchange-writer.test.js new file mode 100644 index 00000000..1fb3aa5e --- /dev/null +++ b/test/plugins/ai-gateway-exchange-writer.test.js @@ -0,0 +1,119 @@ +// @ts-check + +/** + * `recordProjectedExchange` is the seam a live producer that is not the + * proxy writes `ai_gateway_messages` through. What it has to guarantee: + * the rows are the ones the shared expansion produces, and a part some + * other producer already stored is not written twice. + * + * @ref LLP 0252#projection-unchanged [tests]: OTEL is a third producer of the + * dataset, and producer overlap collapses on `part_id` before the write + */ + +import test from 'node:test' +import assert from 'node:assert/strict' + +import { createAiGatewayApi, createGatewayState } from '../../hypaware-core/plugins-workspace/ai-gateway/src/api.js' + +/** + * Storage stub with the surface the dedupe feature-detects: + * `discoverCachePartitions` + `readRows` for committed rows, + * `readSpooledRows` for rows captured but not yet flushed. + * + * @param {{ committed?: string[], spooled?: string[] }} [seed] + */ +function makeStorage(seed = {}) { + const committed = seed.committed ?? [] + const spooled = seed.spooled ?? [] + /** @type {Record[]} */ + const appended = [] + /** @type {string[]} */ + const readRowsCalls = [] + return { + appended, + readRowsCalls, + /** @param {string} dataset @param {string[]} labels */ + cacheTablePath: (dataset, labels) => `/cache/${dataset}/${labels.join('/')}`, + /** @param {{ datasets: string[] }} _scope */ + async discoverCachePartitions(_scope) { + return [{ path: '/cache/committed', partition: {}, rowCount: committed.length }] + }, + /** @param {string} tablePath */ + async *readRows(tablePath) { + readRowsCalls.push(tablePath) + for (const partId of committed) yield { part_id: partId } + }, + async *readSpooledRows() { + for (const partId of spooled) yield { part_id: partId } + }, + /** @param {string} _tablePath @param {unknown} _columns @param {Record[]} rows */ + async appendRows(_tablePath, _columns, rows) { + appended.push(...rows) + }, + } +} + +/** @param {string} sessionId */ +function projection(sessionId) { + return { + provider: 'anthropic', + session_id: sessionId, + client_name: 'claude', + conversation_source: 'claude_code', + messages: [ + { role: 'user', content: 'hello', message_id: 'uuid-user', provider_uuid: 'uuid-user' }, + { role: 'assistant', content: 'hi', message_id: 'uuid-asst', provider_uuid: 'uuid-asst' }, + ], + } +} + +test('a projected exchange becomes rows on the ai_gateway_messages table', async () => { + const storage = makeStorage() + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + const result = await api.recordProjectedExchange(/** @type {any} */ (projection('s1'))) + assert.deepEqual(result, { rowsWritten: 2, rowsSkipped: 0 }) + assert.deepEqual(storage.appended.map((r) => r.part_id), ['uuid-user#0', 'uuid-asst#0']) + assert.equal(storage.appended[0].session_id, 's1') +}) + +test('producer provenance rides the rows', async () => { + const storage = makeStorage() + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + await api.recordProjectedExchange(/** @type {any} */ (projection('s1')), { + gatewayAttributes: { gateway: { source: 'otel' } }, + }) + assert.equal(/** @type {any} */ (storage.appended[0].attributes)?.gateway?.source, 'otel') +}) + +test('a part another producer already committed is skipped, not written twice', async () => { + const storage = makeStorage({ committed: ['uuid-user#0'] }) + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + const result = await api.recordProjectedExchange(/** @type {any} */ (projection('s1'))) + assert.deepEqual(result, { rowsWritten: 1, rowsSkipped: 1 }) + assert.deepEqual(storage.appended.map((r) => r.part_id), ['uuid-asst#0']) +}) + +test('a part still pending in the spool counts as stored', async () => { + const storage = makeStorage({ spooled: ['uuid-asst#0'] }) + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + const result = await api.recordProjectedExchange(/** @type {any} */ (projection('s1'))) + assert.deepEqual(result, { rowsWritten: 1, rowsSkipped: 1 }) + assert.deepEqual(storage.appended.map((r) => r.part_id), ['uuid-user#0']) +}) + +test('re-delivering the same exchange writes nothing the second time', async () => { + const storage = makeStorage() + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + await api.recordProjectedExchange(/** @type {any} */ (projection('s1'))) + const again = await api.recordProjectedExchange(/** @type {any} */ (projection('s1'))) + assert.deepEqual(again, { rowsWritten: 0, rowsSkipped: 0 }) + assert.equal(storage.appended.length, 2) +}) + +test('recording without a storage service fails loudly', async () => { + const api = createAiGatewayApi(createGatewayState()) + await assert.rejects( + () => api.recordProjectedExchange(/** @type {any} */ (projection('s1'))), + /storage service/ + ) +}) diff --git a/test/plugins/ai-gateway-proxy-routing.test.js b/test/plugins/ai-gateway-proxy-routing.test.js index f13bf3c6..6212022a 100644 --- a/test/plugins/ai-gateway-proxy-routing.test.js +++ b/test/plugins/ai-gateway-proxy-routing.test.js @@ -4,7 +4,7 @@ import assert from 'node:assert/strict' import http from 'node:http' import test from 'node:test' -import { createControlHandler } from '../../hypaware-core/plugins-workspace/ai-gateway/src/control.js' +import { createControlHandler } from '../../src/core/control/session_ignore.js' import { compileUpstreams, forwardHeaders, diff --git a/test/plugins/ai-gateway-session-both-recorders.test.js b/test/plugins/ai-gateway-session-both-recorders.test.js new file mode 100644 index 00000000..21c11717 --- /dev/null +++ b/test/plugins/ai-gateway-session-both-recorders.test.js @@ -0,0 +1,273 @@ +// @ts-check + +import assert from 'node:assert/strict' +import fs from 'node:fs' +import http from 'node:http' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { createControlHandler } from '../../src/core/control/session_ignore.js' +import { writePidFile } from '../../src/core/daemon/pid.js' +import { writeStatusFile } from '../../src/core/daemon/status.js' +import { + runSessionIgnore, + runSessionUnignore, +} from '../../hypaware-core/plugins-workspace/ai-gateway/src/session_command.js' + +/** + * With the claude telemetry listener recording Claude Code sessions, "don't + * record this conversation" has to reach BOTH recorders, and only a receipt + * naming each write can support that claim. These tests pin the discovery + * (the listener advertises `control_routes` in the live daemon snapshot and + * is addressed by that advertisement alone), the both-sets outcome, the + * receipt shape (legacy top-level fields stay the gateway's; every + * recorder's outcome rides in `recorders`), and the partial-failure rule + * (an addressed recorder that refuses makes the verb report partial and + * exit unknown, never read as done). + * + * @ref LLP 0256#cli-posts-to-both [tests]: the mutations address every + * listener that offers the route, report each outcome, and a partial + * success is reported, not swallowed. + */ + +const SESSION = 'sess-both-recorders' + +test('ignore lands the id in both recorders and the receipt reports each write', async () => { + const gatewaySet = /** @type {Set} */ (new Set()) + const listenerSet = /** @type {Set} */ (new Set()) + await withControlServer(gatewaySet, async (gatewayBase) => { + await withControlServer(listenerSet, async (listenerBase) => { + const home = daemonHome({ gatewayBase, listenerBase }) + const env = { HYP_HOME: home, CLAUDE_CODE_SESSION_ID: SESSION } + + const json = fakeCtx({ env }) + assert.equal(await runSessionIgnore(['--json'], json.ctx), 0) + assert.ok(gatewaySet.has(SESSION), 'the gateway set holds the id') + assert.ok(listenerSet.has(SESSION), 'the listener set holds the id too') + + const out = JSON.parse(json.stdout()) + assert.equal(out.status, 'ok') + assert.equal(out.guarantee, 'set_membership') + // Legacy top-level fields keep describing the gateway, so existing + // consumers of the receipt lose nothing. + assert.equal(out.ignored, true) + assert.equal(out.endpoint, gatewayBase) + assert.equal(out.endpoint_source, 'daemon_status') + assert.equal(out.endpoint_authenticated, false) + // And the whole write is visible beside them. + assert.equal(out.recorders.length, 2) + const [gw, listener] = out.recorders + assert.deepEqual(gw, { + recorder: 'gateway', + endpoint: gatewayBase, + endpoint_source: 'daemon_status', + endpoint_authenticated: false, + status: 'ok', + ignored: true, + total: 1, + }) + assert.deepEqual(listener, { + recorder: 'claude-telemetry', + endpoint: listenerBase, + endpoint_source: 'daemon_status', + endpoint_authenticated: false, + status: 'ok', + ignored: true, + total: 1, + }) + + // The human receipt names the second write and discloses the trust + // contract for BOTH endpoints (LLP 0166 is per responder). + const human = fakeCtx({ env }) + assert.equal(await runSessionIgnore([], human.ctx), 0) + assert.match(human.stdout(), /session sess-both-recorders: ignored - this id is in the gateway drop set/) + assert.match(human.stdout(), /also claude-telemetry at .*: ignored - this id is in its drop set/) + const trustNotes = human.stdout().match(/nothing proves the responder/g) ?? [] + assert.equal(trustNotes.length, 2, 'one trust disclosure per addressed endpoint') + assert.ok(human.stdout().includes(listenerBase), 'the listener endpoint is named') + }) + }) +}) + +test('unignore removes the id from both recorders', async () => { + const gatewaySet = new Set([SESSION]) + const listenerSet = new Set([SESSION]) + await withControlServer(gatewaySet, async (gatewayBase) => { + await withControlServer(listenerSet, async (listenerBase) => { + const home = daemonHome({ gatewayBase, listenerBase }) + const ctx = fakeCtx({ env: { HYP_HOME: home, CLAUDE_CODE_SESSION_ID: SESSION } }) + assert.equal(await runSessionUnignore(['--json'], ctx.ctx), 0) + assert.equal(gatewaySet.has(SESSION), false) + assert.equal(listenerSet.has(SESSION), false) + const out = JSON.parse(ctx.stdout()) + assert.equal(out.status, 'ok') + assert.equal(out.recorders.length, 2) + assert.ok(out.recorders.every((/** @type {any} */ r) => r.status === 'ok' && r.ignored === false)) + }) + }) +}) + +test('an addressed recorder that refuses makes the write partial, reported and exit-unknown', async () => { + const gatewaySet = /** @type {Set} */ (new Set()) + await withControlServer(gatewaySet, async (gatewayBase) => { + await withRefusingServer(async (listenerBase) => { + const home = daemonHome({ gatewayBase, listenerBase }) + const ctx = fakeCtx({ env: { HYP_HOME: home, CLAUDE_CODE_SESSION_ID: SESSION } }) + const code = await runSessionIgnore(['--json'], ctx.ctx) + + // The gateway write happened and is reported; the listener's refusal + // means the session is STILL being recorded there, so the verb must + // not read as done. + assert.equal(code, 3, 'partial success exits unknown') + assert.ok(gatewaySet.has(SESSION), 'the successful write is kept, not rolled back') + const out = JSON.parse(ctx.stdout()) + assert.equal(out.status, 'partial') + assert.equal(out.recorders.length, 2) + assert.equal(out.recorders[0].status, 'ok') + assert.equal(out.recorders[1].status, 'error') + assert.match(out.recorders[1].error, /HTTP 500/) + assert.match(ctx.stderr(), /claude-telemetry at .*: /, 'the failure names the recorder') + }) + }) +}) + +test('with no advertisement the receipt is the single-recorder one', async () => { + const gatewaySet = /** @type {Set} */ (new Set()) + await withControlServer(gatewaySet, async (gatewayBase) => { + const home = daemonHome({ gatewayBase }) + const ctx = fakeCtx({ env: { HYP_HOME: home, CLAUDE_CODE_SESSION_ID: SESSION } }) + assert.equal(await runSessionIgnore(['--json'], ctx.ctx), 0) + const out = JSON.parse(ctx.stdout()) + assert.equal(out.status, 'ok') + assert.equal(out.recorders.length, 1) + assert.equal(out.recorders[0].recorder, 'gateway') + }) +}) + +test('an advertisement naming the gateway\'s own endpoint is not addressed twice', async () => { + // Belt for a future recorder riding the gateway's listener: the gateway is + // already a target through its own resolution, so the same endpoint must + // not receive the mutation twice. + const gatewaySet = /** @type {Set} */ (new Set()) + let hits = 0 + await withControlServer(gatewaySet, async (gatewayBase) => { + const home = daemonHome({ gatewayBase, listenerBase: gatewayBase }) + const ctx = fakeCtx({ env: { HYP_HOME: home, CLAUDE_CODE_SESSION_ID: SESSION } }) + assert.equal(await runSessionIgnore(['--json'], ctx.ctx), 0) + const out = JSON.parse(ctx.stdout()) + assert.equal(out.recorders.length, 1) + }, () => { hits += 1 }) + assert.equal(hits, 1, 'one POST reached the shared endpoint') +}) + +/* ------------------------------------------------------------------ */ +/* helpers */ +/* ------------------------------------------------------------------ */ + +/** + * The genuine control route over a shared set, exactly as both recorders + * host it. + * + * @param {Set} set + * @param {(base: string) => Promise} fn + * @param {() => void} [onRequest] + */ +async function withControlServer(set, fn, onRequest) { + const handler = createControlHandler({ ignoredSessions: set }) + const server = http.createServer((req, res) => { + onRequest?.() + const url = new URL(req.url ?? '/', 'http://127.0.0.1') + handler(req, res, url) + }) + await new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve(undefined))) + const addr = server.address() + const port = typeof addr === 'object' && addr ? addr.port : 0 + try { + await fn(`http://127.0.0.1:${port}`) + } finally { + await new Promise((resolve) => server.close(() => resolve(undefined))) + } +} + +/** + * A recorder that is RUNNING and refuses: bound, answering, and unable to + * take the write. Distinct from not-running (which is never addressed). + * + * @param {(base: string) => Promise} fn + */ +async function withRefusingServer(fn) { + const server = http.createServer((req, res) => { + req.resume() + res.writeHead(500, { 'content-type': 'application/json' }) + res.end(JSON.stringify({ error: 'wedged' })) + }) + await new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve(undefined))) + const addr = server.address() + const port = typeof addr === 'object' && addr ? addr.port : 0 + try { + await fn(`http://127.0.0.1:${port}`) + } finally { + await new Promise((resolve) => server.close(() => resolve(undefined))) + } +} + +/** + * A `HYP_HOME` whose live daemon snapshot names the gateway's bound port + * and, when `listenerBase` is given, a claude-telemetry source advertising + * the session-ignore control route at its own bound listener - the exact + * shape the daemon writes. + * + * @param {{ gatewayBase: string, listenerBase?: string }} args + * @returns {string} + */ +function daemonHome({ gatewayBase, listenerBase }) { + const gatewayUrl = new URL(gatewayBase) + const hypHome = fs.mkdtempSync(path.join(os.tmpdir(), 'hyp-session-both-')) + const stateRoot = path.join(hypHome, 'hypaware') + fs.mkdirSync(path.join(stateRoot, 'run'), { recursive: true }) + writePidFile(stateRoot, /** @type {any} */ ({ pid: process.pid, runId: 'test-run', mode: 'foreground' })) + const sources = [ + { + name: 'ai-gateway', + plugin: '@hypaware/ai-gateway', + state: 'ready', + details: { host: gatewayUrl.hostname, port: Number(gatewayUrl.port) }, + }, + ] + if (listenerBase) { + const listenerUrl = new URL(listenerBase) + sources.push({ + name: 'claude-telemetry', + plugin: '@hypaware/claude', + state: 'ready', + details: /** @type {any} */ ({ + listen_host: listenerUrl.hostname, + listen_port: Number(listenerUrl.port), + control_routes: ['ignore/session'], + }), + }) + } + writeStatusFile(stateRoot, /** @type {any} */ ({ state: 'running', sources, sinks: [] })) + return hypHome +} + +/** + * @param {{ env?: Record }} args + */ +function fakeCtx(args) { + let out = '' + let err = '' + const hypHome = args.env?.HYP_HOME ?? fs.mkdtempSync(path.join(os.tmpdir(), 'hyp-session-home-')) + const ctx = { + stdout: { write: (/** @type {string} */ s) => { out += s; return true } }, + stderr: { write: (/** @type {string} */ s) => { err += s; return true } }, + env: { HYP_HOME: hypHome, ...(args.env ?? {}) }, + cwd: '/repo/here', + config: { + version: 2, + plugins: [{ name: '@hypaware/ai-gateway' }, { name: '@hypaware/claude' }], + }, + } + return { ctx: /** @type {any} */ (ctx), stdout: () => out, stderr: () => err } +} diff --git a/test/plugins/ai-gateway-session-ignore-receipt.test.js b/test/plugins/ai-gateway-session-ignore-receipt.test.js index cb7c1e5f..03fc3399 100644 --- a/test/plugins/ai-gateway-session-ignore-receipt.test.js +++ b/test/plugins/ai-gateway-session-ignore-receipt.test.js @@ -8,7 +8,7 @@ import path from 'node:path' import test from 'node:test' import { fileURLToPath } from 'node:url' -import { createControlHandler } from '../../hypaware-core/plugins-workspace/ai-gateway/src/control.js' +import { createControlHandler } from '../../src/core/control/session_ignore.js' import { createCodexExchangeProjector } from '../../hypaware-core/plugins-workspace/codex/src/exchange-projector.js' import { USAGE_POLICY_DROP } from '../../src/core/usage-policy/index.js' import { runSessionIgnore, runSessionStatus, runSessionUnignore } from '../../hypaware-core/plugins-workspace/ai-gateway/src/session_command.js' @@ -21,7 +21,7 @@ import { runSessionIgnore, runSessionStatus, runSessionUnignore } from '../../hy * Regression suite for issue #460: the `POST` receipt claimed a drop it could * not have verified. * - * `control.js` adds an opaque token to a `Set` and answers `ignored: true` for + * The shared control handler adds an opaque token to a `Set` and answers `ignored: true` for * whatever it was handed; the drop happens later, in the client adapter, keyed * on the `session_id` that adapter stamps on the row. Nothing compares the two, * so the receipt is evidence of a write and of nothing else - yet diff --git a/test/plugins/ai-gateway-session-responder-trust.test.js b/test/plugins/ai-gateway-session-responder-trust.test.js index f1095b23..fd4d2e9a 100644 --- a/test/plugins/ai-gateway-session-responder-trust.test.js +++ b/test/plugins/ai-gateway-session-responder-trust.test.js @@ -7,7 +7,7 @@ import os from 'node:os' import path from 'node:path' import test from 'node:test' -import { createControlHandler } from '../../hypaware-core/plugins-workspace/ai-gateway/src/control.js' +import { createControlHandler } from '../../src/core/control/session_ignore.js' import { writePidFile } from '../../src/core/daemon/pid.js' import { writeStatusFile } from '../../src/core/daemon/status.js' import { diff --git a/test/plugins/ai-gateway-session-status.test.js b/test/plugins/ai-gateway-session-status.test.js index 679a271f..2a52fafe 100644 --- a/test/plugins/ai-gateway-session-status.test.js +++ b/test/plugins/ai-gateway-session-status.test.js @@ -7,7 +7,7 @@ import os from 'node:os' import path from 'node:path' import test from 'node:test' -import { createControlHandler } from '../../hypaware-core/plugins-workspace/ai-gateway/src/control.js' +import { createControlHandler } from '../../src/core/control/session_ignore.js' import { createCodexExchangeProjector } from '../../hypaware-core/plugins-workspace/codex/src/exchange-projector.js' import { USAGE_POLICY_DROP } from '../../src/core/usage-policy/index.js' import { diff --git a/test/plugins/capture-seam-machine-local-list.test.js b/test/plugins/capture-seam-machine-local-list.test.js index 780f8100..425c76bc 100644 --- a/test/plugins/capture-seam-machine-local-list.test.js +++ b/test/plugins/capture-seam-machine-local-list.test.js @@ -279,6 +279,7 @@ async function claudeProjectorViaActivate(env) { skills: { register() {} }, agents: { register() {} }, initPresets: { register() {} }, + query: { registerDataset() {} }, }) await activateClaude(ctx) assert.ok(projector, 'claude activate() registered an exchange projector') diff --git a/test/plugins/claude-hook-spool-cap.test.js b/test/plugins/claude-hook-spool-cap.test.js new file mode 100644 index 00000000..624f64bb --- /dev/null +++ b/test/plugins/claude-hook-spool-cap.test.js @@ -0,0 +1,280 @@ +// @ts-check + +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { Readable } from 'node:stream' +import test from 'node:test' + +import { runClaudeSessionContextHook } from '../../hypaware-core/plugins-workspace/claude/src/hook_command.js' +import { readSessionContext } from '../../hypaware-core/plugins-workspace/claude/src/session_context.js' +import { + DEFAULT_SPOOL_MAX_BYTES, + claudeBodySpoolDir, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/spool.js' + +/** + * The body spool's byte cap, enforced from OUTSIDE the daemon. + * + * LLP 0253 #byte-cap names the daemon-down window as the reason the cap + * exists, but shipped every enforcement inside the listener source, so that + * window was the one nothing swept. LLP 0263 makes the client hook the second + * enforcer. These tests drive the hook with no daemon anywhere in sight, + * which is the whole point: if they pass, a machine whose daemon crashed, + * was never started, or was uninstalled without a detach still converges to + * the cap. + * + * @ref LLP 0263#hook-enforces-the-cap [tests]: the hook bounds the spool with + * no daemon running + */ + +test('the hook evicts oldest-first when the spool is over the operator cap, with no daemon anywhere', async () => { + const env = await stageEnv() + try { + // Three 100-byte bodies, oldest to newest, against a 250-byte cap: the + // oldest must go and the two newest must survive (LLP 0253 keeps the + // bodies whose events are still arriving). + await writeBody(env.spoolDir, 'old.json', 100, 1_000) + await writeBody(env.spoolDir, 'middle.json', 100, 2_000) + await writeBody(env.spoolDir, 'new.json', 100, 3_000) + + const code = await runHook(env, { spoolMaxBytes: 250 }) + assert.equal(code, 0) + + const left = (await fs.readdir(env.spoolDir)).sort() + assert.deepEqual(left, ['middle.json', 'new.json'], 'the oldest body is evicted, the newest are kept') + assert.ok(await totalBytes(env.spoolDir) <= 250, 'the spool is back under the cap') + } finally { + await env.cleanup() + } +}) + +// @ref LLP 0263#hook-enforces-the-cap [tests]: the cap the hook applies is the +// operator's `telemetry.spool_max_bytes`, never a rule of the hook's own. +test('the hook reads the operator cap out of the @hypaware/claude config slice', async () => { + const env = await stageEnv() + try { + await writeBody(env.spoolDir, 'a.json', 100, 1_000) + await writeBody(env.spoolDir, 'b.json', 100, 2_000) + + // 150 bytes is far below the 512 MB default: if the hook ignored the + // config slice, nothing here would be evicted at all. + assert.ok(150 < DEFAULT_SPOOL_MAX_BYTES, 'the test cap is below the default it must override') + const code = await runHook(env, { spoolMaxBytes: 150 }) + assert.equal(code, 0) + + assert.deepEqual(await fs.readdir(env.spoolDir), ['b.json'], 'the configured cap bound the sweep') + } finally { + await env.cleanup() + } +}) + +test('a malformed cap falls back to the default rather than evicting on a bad number', async () => { + const env = await stageEnv() + try { + await writeBody(env.spoolDir, 'a.json', 100, 1_000) + for (const bad of ['not-a-number', -1, 0, 1.5, null]) { + const code = await runHook(env, { spoolMaxBytes: /** @type {any} */ (bad) }) + assert.equal(code, 0) + assert.deepEqual( + await fs.readdir(env.spoolDir), + ['a.json'], + `a ${JSON.stringify(bad)} cap falls back to the default, which evicts nothing here` + ) + } + } finally { + await env.cleanup() + } +}) + +test('an under-cap spool is left alone', async () => { + const env = await stageEnv() + try { + await writeBody(env.spoolDir, 'a.json', 100, 1_000) + await writeBody(env.spoolDir, 'b.json', 100, 2_000) + + const code = await runHook(env, { spoolMaxBytes: 10_000 }) + assert.equal(code, 0) + assert.deepEqual((await fs.readdir(env.spoolDir)).sort(), ['a.json', 'b.json']) + } finally { + await env.cleanup() + } +}) + +// A proxy-attached or unattached machine has no spool directory at all. The +// sweep must be a silent no-op there, not an error the hook has to swallow. +test('a machine with no spool directory sweeps to a no-op and still records context', async () => { + const env = await stageEnv({ createSpool: false }) + try { + const code = await runHook(env, { spoolMaxBytes: 100 }) + assert.equal(code, 0) + + const records = await readSessionContext(env.stateFile) + assert.ok(records.length >= 1, 'the session-context record still landed') + await assert.rejects(fs.stat(env.spoolDir), 'the sweep did not create the directory it found missing') + } finally { + await env.cleanup() + } +}) + +// @ref LLP 0263#never-interrupts [tests]: the sweep runs on invocations that +// record nothing. A missing --state-file says nothing about whether Claude +// Code is filling the spool, and this is exactly the path a half-configured +// machine takes. +test('the cap is enforced even when the event records no context at all', async () => { + const env = await stageEnv() + try { + await writeBody(env.spoolDir, 'old.json', 100, 1_000) + await writeBody(env.spoolDir, 'new.json', 100, 2_000) + + // No --state-file: recordSessionContext bails immediately. + const code = await runClaudeSessionContextHook( + ['session-context'], + ctxFor(env, { spoolMaxBytes: 150 }, { session_id: 's', cwd: '/w' }), + { gitBranch: async () => undefined, gitRepoFacts: async () => ({}) } + ) + assert.equal(code, 0) + assert.deepEqual(await fs.readdir(env.spoolDir), ['new.json'], 'the spool was swept anyway') + } finally { + await env.cleanup() + } +}) + +test('a malformed event still sweeps the spool', async () => { + const env = await stageEnv() + try { + await writeBody(env.spoolDir, 'old.json', 100, 1_000) + await writeBody(env.spoolDir, 'new.json', 100, 2_000) + + const code = await runClaudeSessionContextHook( + ['session-context', '--state-file', env.stateFile], + ctxFor(env, { spoolMaxBytes: 150 }, 'not json at all'), + { gitBranch: async () => undefined, gitRepoFacts: async () => ({}) } + ) + assert.equal(code, 0) + assert.deepEqual(await fs.readdir(env.spoolDir), ['new.json']) + } finally { + await env.cleanup() + } +}) + +// @ref LLP 0263#never-interrupts [tests]: a sweep failure is swallowed. The +// hook must never throw back into Claude Code, and a spool it cannot read is +// not a reason to lose the session-context record. +test('a throwing sweep never fails the hook and never costs the context record', async () => { + const env = await stageEnv() + try { + const code = await runClaudeSessionContextHook( + ['session-context', '--state-file', env.stateFile], + ctxFor(env, {}, { session_id: 'sess-throw', cwd: '/work/repo' }), + { + gitBranch: async () => undefined, + gitRepoFacts: async () => ({}), + sweepSpool: async () => { throw new Error('spool unreadable') }, + } + ) + assert.equal(code, 0) + + const records = await readSessionContext(env.stateFile) + assert.equal(records.length, 1) + assert.equal(records[0].cwd, '/work/repo') + } finally { + await env.cleanup() + } +}) + +// @ref LLP 0263#never-interrupts [tests]: ordering. LLP 0085 exists to shrink +// the window where the projector reads a cwd-less record, so the sweep may +// never run ahead of the appends. +test('the sweep runs after the session-context records, never before', async () => { + const env = await stageEnv() + try { + let recordsAtSweepTime = /** @type {any[] | null} */ (null) + const code = await runClaudeSessionContextHook( + ['session-context', '--state-file', env.stateFile], + ctxFor(env, {}, { session_id: 'sess-order', cwd: '/work/repo' }), + { + gitBranch: async () => 'main', + gitRepoFacts: async () => ({ repoRoot: '/work/repo' }), + sweepSpool: async () => { recordsAtSweepTime = await readSessionContext(env.stateFile) }, + } + ) + assert.equal(code, 0) + assert.ok(recordsAtSweepTime, 'the sweep ran') + assert.equal(recordsAtSweepTime.length, 2, 'both records were durable before the sweep started') + assert.equal(recordsAtSweepTime[1].git_branch, 'main', 'even the enriched record landed first') + } finally { + await env.cleanup() + } +}) + +/** + * @param {{ createSpool?: boolean }} [opts] + */ +async function stageEnv(opts = {}) { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'claude-hook-spool-cap-')) + const hypHome = path.join(homeDir, '.hyp') + const stateDir = path.join(hypHome, 'state', '@hypaware-claude') + await fs.mkdir(stateDir, { recursive: true }) + const spoolDir = claudeBodySpoolDir(hypHome) + if (opts.createSpool !== false) await fs.mkdir(spoolDir, { recursive: true, mode: 0o700 }) + return { + hypHome, + spoolDir, + stateFile: path.join(stateDir, 'session-context.jsonl'), + cleanup: async () => { await fs.rm(homeDir, { recursive: true, force: true }) }, + } +} + +/** + * @param {{ hypHome: string, stateFile: string }} env + * @param {{ spoolMaxBytes?: number }} pluginConfig + * @param {unknown} event + */ +function ctxFor(env, pluginConfig, event) { + const telemetry = pluginConfig.spoolMaxBytes === undefined + ? {} + : { telemetry: { spool_max_bytes: pluginConfig.spoolMaxBytes } } + return /** @type {any} */ ({ + stdout: { write() { return true } }, + stderr: { write() { return true } }, + stdin: /** @type {NodeJS.ReadStream} */ ( + Readable.from([typeof event === 'string' ? event : JSON.stringify(event)]) + ), + env: { ...process.env, HYP_HOME: env.hypHome }, + config: { plugins: [{ name: '@hypaware/claude', config: telemetry }] }, + }) +} + +/** + * @param {{ hypHome: string, stateFile: string }} env + * @param {{ spoolMaxBytes?: number }} pluginConfig + */ +function runHook(env, pluginConfig) { + return runClaudeSessionContextHook( + ['session-context', '--state-file', env.stateFile], + ctxFor(env, pluginConfig, { session_id: 'sess-spool', cwd: '/work/repo' }), + { gitBranch: async () => undefined, gitRepoFacts: async () => ({}) } + ) +} + +/** + * @param {string} dir + * @param {string} name + * @param {number} size + * @param {number} mtimeSeconds + */ +async function writeBody(dir, name, size, mtimeSeconds) { + const file = path.join(dir, name) + await fs.writeFile(file, 'x'.repeat(size)) + await fs.utimes(file, mtimeSeconds, mtimeSeconds) +} + +/** @param {string} dir */ +async function totalBytes(dir) { + const names = await fs.readdir(dir) + let sum = 0 + for (const name of names) sum += (await fs.stat(path.join(dir, name))).size + return sum +} diff --git a/test/plugins/claude-otel-migration.test.js b/test/plugins/claude-otel-migration.test.js new file mode 100644 index 00000000..a148e6d0 --- /dev/null +++ b/test/plugins/claude-otel-migration.test.js @@ -0,0 +1,449 @@ +// @ts-check + +/** + * The proxy-to-otel migration `hyp attach claude` performs on a machine that + * is still proxy-attached (LLP 0262 #migration): the settings write flips the + * marker and releases the proxy keys through the ordinary mode-switch rule, + * the launchd environment is unwound, and the CA trust is OFFERED as + * `hyp detach claude --purge` but never taken. These tests drive the real + * adapter (through `activate()`), the way an attach reaches it in production; + * the writer-level key release itself is pinned by + * claude-settings-otel-attach.test.js. + * + * @ref LLP 0262#migration [tests]: one command migrates a proxy-attached machine, and the CA purge is offered, never forced + */ + +import assert from 'node:assert/strict' +import fsp from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { + activate as activateClaude, + unwindProxyLaunchdEnv, +} from '../../hypaware-core/plugins-workspace/claude/src/index.js' +import { + MODE_BASE_URL, + MODE_OTEL, + MODE_PROXY, + attach as writeSettings, + otelModeEnv, +} from '../../hypaware-core/plugins-workspace/claude/src/settings.js' +import { ensureLocalCa } from '../../src/core/tls/ca.js' +import { collectHypAwareStatus, probeClientAttachFromDescriptor } from '../../src/core/daemon/status.js' +import { renderStatusText } from '../../src/core/commands/status.js' + +const GATEWAY_PORT = 18521 +const ENDPOINT = `http://127.0.0.1:${GATEWAY_PORT}` + +/** The claude descriptor, the shape `hyp status` probes the marker through. */ +const CLAUDE_DESCRIPTOR = /** @type {any} */ ({ + name: 'claude', + plugin: '@hypaware/claude', + attachProbe: { + format: 'json', + settings_file: '.claude/settings.json', + marker_key: '_hypaware', + }, +}) + +/** + * A temp home seeded with a user-owned settings file, plus the activation + * fixture that registers the real claude adapter against a fake gateway. + * + * @param {{ claudeVersion?: string }} [opts] + */ +async function rig(opts = {}) { + const root = await fsp.mkdtemp(path.join(os.tmpdir(), 'hyp-otel-migration-')) + const settingsPath = path.join(root, '.claude', 'settings.json') + await fsp.mkdir(path.dirname(settingsPath), { recursive: true }) + const seed = { + env: { ANTHROPIC_API_KEY: 'sk-user-key' }, + permissions: { allow: ['Bash(ls *)'] }, + } + await fsp.writeFile(settingsPath, JSON.stringify(seed, null, 2) + '\n') + + // The install's own config, so `hyp status` reads this machine the way it + // reads a real one: the claude plugin is enabled, which is what makes the + // client line say `configured` beside the attach state. + await fsp.mkdir(path.join(root, '.hyp'), { recursive: true }) + await fsp.writeFile( + path.join(root, '.hyp', 'hypaware-config.json'), + JSON.stringify({ + version: 2, + plugins: [ + { + name: '@hypaware/ai-gateway', + config: { + upstreams: [{ + name: 'anthropic', + base_url: 'https://api.anthropic.com', + path_prefix: '/v1/messages', + }], + }, + }, + { name: '@hypaware/claude', config: { proxy: '@hypaware/ai-gateway' } }, + ], + }, null, 2) + '\n' + ) + + const env = { + HOME: root, + HYP_HOME: path.join(root, '.hyp'), + HYP_CLAUDE_CODE_VERSION: opts.claudeVersion ?? '2.1.233', + } + + /** @type {any} */ + const gateway = { + registerUpstreamPreset() {}, + registerExchangeProjector() {}, + registerSettlementEnricher() {}, + /** @type {any} */ + client: undefined, + registerClient(/** @type {any} */ client) { this.client = client }, + } + const ctx = /** @type {any} */ ({ + env, + paths: { stateDir: path.join(root, '.hyp', 'hypaware', 'plugins', 'claude') }, + plugin: { version: '0.0.0-test' }, + config: {}, + configRegistry: { registerSection() {} }, + requireCapability: () => gateway, + backfills: { register() {} }, + commands: { register() {} }, + skills: { register() {} }, + agents: { register() {} }, + initPresets: { register() {} }, + sources: { register() {} }, + query: { registerDataset() {} }, + }) + await activateClaude(ctx) + + return { + root, + env, + settingsPath, + stateRoot: path.join(root, '.hyp', 'hypaware'), + gateway, + /** @returns {Promise>} */ + async read() { + return JSON.parse(await fsp.readFile(settingsPath, 'utf8')) + }, + async raw() { + return fsp.readFile(settingsPath, 'utf8') + }, + cleanup: () => fsp.rm(root, { recursive: true, force: true }), + } +} + +/** + * Proxy-attach the rig's settings file the way a proxy-mode install left it, + * with a real CA on disk so "the purge was never run" is observable. + * + * @param {Awaited>} r + */ +async function seedProxyAttach(r) { + const ca = await ensureLocalCa({ stateRoot: r.stateRoot, hosts: ['api.anthropic.com'] }) + await writeSettings({ + port: GATEWAY_PORT, + version: '2.0.0', + stateFile: path.join(r.root, 'session-context.jsonl'), + settingsPath: r.settingsPath, + mode: MODE_PROXY, + caCertPath: ca.certPath, + }) + return ca +} + +function makeBuf() { + let value = '' + return { + write(/** @type {unknown} */ chunk) { + value += String(chunk) + return true + }, + text() { + return value + }, + } +} + +/** + * The `hyp status` client line for this rig, produced the way the command + * produces it: the real collector (which discovers the bundled claude + * descriptor and probes the marker this rig wrote) feeding the real text + * renderer. Nothing about the line is fabricated, so it moves only when a + * real attach moves the marker. + * + * @param {Awaited>} r + * @returns {Promise} + */ +async function statusClientLine(r) { + const report = await collectHypAwareStatus({ + env: { ...process.env, HOME: r.root, HYP_HOME: path.join(r.root, '.hyp'), HYP_CONFIG: '' }, + homeDir: r.root, + }) + const stdout = makeBuf() + renderStatusText({ + report, + // The live gateway's registered clients, which is what the command passes: + // this rig's activation registered the claude adapter on its gateway. + clientNames: ['claude'], + datasets: [], + cacheRoot: path.join(r.root, 'cache'), + stdout, + }) + const line = stdout.text().split('\n').find((l) => l.includes('- claude ')) + assert.ok(line !== undefined, 'hyp status listed no claude client line') + return line +} + +test('hyp attach claude migrates a proxy attach: marker flips, proxy keys release, nothing else is touched', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + const ca = await seedProxyAttach(r) + + const before = await probeClientAttachFromDescriptor({ + descriptor: CLAUDE_DESCRIPTOR, + homeDir: r.root, + env: r.env, + }) + assert.equal(before.mode, 'proxy') + + const buf = makeBuf() + await r.gateway.client.attach({ endpoint: ENDPOINT, stdout: buf, stderr: buf }) + + const value = await r.read() + // The marker is now an otel marker, and the spool it records is swept by + // detach and purge. + assert.equal(value._hypaware.mode, 'otel') + assert.equal(typeof value._hypaware.spool_dir, 'string') + // The proxy keys are gone from env: nothing routes Claude Code any more. + assert.equal(Object.hasOwn(value.env, 'HTTPS_PROXY'), false) + assert.equal(Object.hasOwn(value.env, 'NODE_EXTRA_CA_CERTS'), false) + // Nothing else was touched: the env holds exactly the telemetry block plus + // the user's own key, and the user's other settings survive verbatim. + const expectedKeys = [ + 'ANTHROPIC_API_KEY', + ...otelModeEnv({ telemetryPort: 1, spoolDir: '/x' }).map((e) => e.key), + ].sort() + assert.deepEqual(Object.keys(value.env).sort(), expectedKeys) + assert.equal(value.env.ANTHROPIC_API_KEY, 'sk-user-key') + assert.deepEqual(value.permissions, { allow: ['Bash(ls *)'] }) + + // The migration is narrated, and the CA purge is offered as the detach + // command, not performed: the CA is still on disk afterwards. + const out = buf.text() + assert.match(out, /Migrated from proxy attach/) + assert.match(out, /keep proxying until they restart/) + assert.match(out, /hyp detach claude --purge/) + await fsp.access(ca.certPath) + + // The launchd unwind ran through the real seam. Under the test runner the + // service-manager guard refuses the spawn (LLP 0181), which surfaces as the + // by-hand warning; what matters here is that the attempt was made and the + // attach still succeeded. + if (process.platform === 'darwin') { + assert.match(out, /launchd environment could not be released/) + assert.match(out, /launchctl unsetenv NODE_USE_SYSTEM_CA/) + } + + // hyp status answers from this same probe: the machine now reads as otel. + const after = await probeClientAttachFromDescriptor({ + descriptor: CLAUDE_DESCRIPTOR, + homeDir: r.root, + env: r.env, + }) + assert.equal(after.attached, true) + assert.equal(after.mode, 'otel') +}) + +test('the migration facts ride the --json payload', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + await seedProxyAttach(r) + + const buf = makeBuf() + await r.gateway.client.attach({ endpoint: ENDPOINT, stdout: buf, stderr: buf, json: true }) + + const payload = JSON.parse(buf.text().trim().split('\n')[0]) + assert.equal(payload.status, 'ok') + assert.equal(payload.mode, 'otel') + assert.equal(payload.migrated_from, 'proxy') + if (process.platform === 'darwin') { + // The guard refused the real launchctl under the test runner, so the + // unwind reports false rather than being silently absent. + assert.equal(payload.launchd_env_removed, false) + } +}) + +// The migration is only finished when the surface a human checks agrees. The +// probe above is one half of `hyp status`; this drives both halves end to end +// over the same machine, before and after the one command. +// @ref LLP 0262#migration [tests]: hyp status reflects the new attach mode after the migration +test('hyp status reads the migrated machine as otel-attached', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + await seedProxyAttach(r) + + assert.match(await statusClientLine(r), /- claude {2}\[configured, attached \(proxy\)\]/) + + const buf = makeBuf() + await r.gateway.client.attach({ endpoint: ENDPOINT, stdout: buf, stderr: buf }) + + assert.match(await statusClientLine(r), /- claude {2}\[configured, attached \(otel\)\]/) +}) + +// Below the floor the migration must not begin: the proxy attach keeps +// working exactly as it is, and no residue is unwound for a switch that never +// happened. +// @ref LLP 0258#version-floor [tests]: a refusal on a proxy-attached machine leaves the proxy attach byte for byte +test('a floor refusal leaves the proxy attach untouched and unwinds nothing', async (t) => { + const r = await rig({ claudeVersion: '2.1.100' }) + t.after(() => r.cleanup()) + const ca = await seedProxyAttach(r) + const before = await r.raw() + + const buf = makeBuf() + await assert.rejects( + () => r.gateway.client.attach({ endpoint: ENDPOINT, stdout: buf, stderr: buf }), + /claude update/ + ) + assert.equal(await r.raw(), before) + assert.equal((await r.read())._hypaware.mode, 'proxy') + await fsp.access(ca.certPath) + assert.doesNotMatch(buf.text(), /Migrated from proxy attach/) + // The residue unwind is downstream of the settings write, so a refusal + // never reaches it: on darwin an attempted release would have printed here. + assert.doesNotMatch(buf.text(), /launchd/) +}) + +test('a re-attach after the migration is routine: no migration notes, no offer', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + await seedProxyAttach(r) + + const first = makeBuf() + await r.gateway.client.attach({ endpoint: ENDPOINT, stdout: first, stderr: first }) + assert.match(first.text(), /Migrated from proxy attach/) + + const second = makeBuf() + await r.gateway.client.attach({ endpoint: ENDPOINT, stdout: second, stderr: second }) + assert.doesNotMatch(second.text(), /Migrated from proxy attach/) + assert.doesNotMatch(second.text(), /hyp detach claude --purge/) + assert.doesNotMatch(second.text(), /launchd/) + assert.equal((await r.read())._hypaware.mode, 'otel') +}) + +// Only a proxy attach has residue outside the settings file; a base-URL +// attach migrates through the mode-switch key release alone. +test('a base-URL attach switches to otel without migration notes', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + await writeSettings({ + port: GATEWAY_PORT, + version: '2.0.0', + stateFile: path.join(r.root, 'session-context.jsonl'), + settingsPath: r.settingsPath, + mode: MODE_BASE_URL, + }) + + const buf = makeBuf() + await r.gateway.client.attach({ endpoint: ENDPOINT, stdout: buf, stderr: buf }) + + const value = await r.read() + assert.equal(value._hypaware.mode, 'otel') + assert.equal(Object.hasOwn(value.env, 'ANTHROPIC_BASE_URL'), false) + assert.doesNotMatch(buf.text(), /Migrated from proxy attach/) + // No proxy attach means no residue, so the launchd environment is left + // alone: a base-URL machine may never have had it set at all. + assert.doesNotMatch(buf.text(), /launchd/) +}) + +test('the writer reports the prior marker mode for the adapter to act on', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + await seedProxyAttach(r) + + const result = await writeSettings({ + port: GATEWAY_PORT, + version: '2.0.0', + stateFile: path.join(r.root, 'session-context.jsonl'), + settingsPath: r.settingsPath, + mode: MODE_OTEL, + telemetryPort: 4319, + spoolDir: path.join(r.root, '.hyp', 'spool', 'claude-bodies'), + claudeVersion: '2.1.233', + }) + assert.equal(result.changed && result.priorMode, 'proxy') +}) + +test('a first attach reports no prior mode', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + const result = await writeSettings({ + port: GATEWAY_PORT, + version: '2.0.0', + stateFile: path.join(r.root, 'session-context.jsonl'), + settingsPath: r.settingsPath, + mode: MODE_OTEL, + telemetryPort: 4319, + spoolDir: path.join(r.root, '.hyp', 'spool', 'claude-bodies'), + claudeVersion: '2.1.233', + }) + assert.equal(result.changed && 'priorMode' in result, false) +}) + +// The unwind helper itself, with the launchctl seam injected: this is the +// deterministic proof the migration invokes the unwind on macOS and never +// touches launchctl anywhere else. +test('unwindProxyLaunchdEnv removes the launchd env on darwin', async () => { + /** @type {unknown[]} */ + const calls = [] + const result = await unwindProxyLaunchdEnv({ + homeDir: '/tmp/some-home', + platform: 'darwin', + removeEnv: async (args) => { + calls.push(args) + return { unset: true, removedPlist: true } + }, + }) + assert.deepEqual(result, { launchdEnvRemoved: true, warnings: [] }) + assert.deepEqual(calls, [{ homeDir: '/tmp/some-home' }]) +}) + +test('unwindProxyLaunchdEnv never runs launchctl off darwin', async () => { + const result = await unwindProxyLaunchdEnv({ + homeDir: '/tmp/some-home', + platform: 'linux', + removeEnv: async () => { + throw new Error('must not be called') + }, + }) + assert.deepEqual(result, { warnings: [] }) +}) + +test('unwindProxyLaunchdEnv degrades a failed unset to the by-hand hint', async () => { + const result = await unwindProxyLaunchdEnv({ + platform: 'darwin', + removeEnv: async () => ({ unset: false, removedPlist: false, detail: 'exit 1' }), + }) + assert.equal(result.launchdEnvRemoved, false) + assert.match(result.warnings[0], /NODE_USE_SYSTEM_CA could not be unset/) + assert.match(result.warnings[0], /exit 1/) + assert.match(result.warnings[0], /launchctl unsetenv NODE_USE_SYSTEM_CA/) +}) + +test('unwindProxyLaunchdEnv degrades a thrown release to the by-hand hint', async () => { + const result = await unwindProxyLaunchdEnv({ + platform: 'darwin', + removeEnv: async () => { + throw new Error('sandbox says no') + }, + }) + assert.equal(result.launchdEnvRemoved, false) + assert.match(result.warnings[0], /launchd environment could not be released/) + assert.match(result.warnings[0], /sandbox says no/) +}) diff --git a/test/plugins/claude-otel-proxy-overlap.test.js b/test/plugins/claude-otel-proxy-overlap.test.js new file mode 100644 index 00000000..300d09d2 --- /dev/null +++ b/test/plugins/claude-otel-proxy-overlap.test.js @@ -0,0 +1,268 @@ +// @ts-check + +/** + * The migration overlap window (LLP 0262 #migration): sessions started + * before the mode flip keep proxying until they restart while new events + * arrive over OTEL, so for a while BOTH producers capture the same session. + * That is harmless only if the two producers agree on row identity, and + * these tests pin both halves of that promise: + * + * 1. the proxy projector (native transcript identity) and the telemetry + * projection (`message.uuid` identity) yield the SAME `part_id`s for the + * same session content, and + * 2. the OTEL producer's pre-write dedupe drops every part the proxy + * already stored, so the overlap lands as one row set. + * + * The mirror direction needs no separate fixture: the proxy's flush-time + * dedupe asks the same committed-`part_id` membership question + * (`dedupeByPartId` in dataset.js), so identical part identity is what makes + * either arrival order collapse. + * + * @ref LLP 0262#migration [tests]: a session captured by both producers dedupes to one row set + * @ref LLP 0252#projection-unchanged [tests]: producer overlap collapses on part_id before the write + */ + +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { + aiGatewayRowsFromProjectedExchange, + createAiGatewayMessageProjector, +} from '../../hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js' +import { createAiGatewayApi, createGatewayState } from '../../hypaware-core/plugins-workspace/ai-gateway/src/api.js' +import { createClaudeExchangeProjector } from '../../hypaware-core/plugins-workspace/claude/src/projector.js' +import { flattenClaudeTelemetryEvents } from '../../hypaware-core/plugins-workspace/claude/src/telemetry/events.js' +import { projectClaudeTelemetryEvents } from '../../hypaware-core/plugins-workspace/claude/src/telemetry/projection.js' + +const SESSION = 'e53c128d-9f45-470f-86f1-d5b5f3766708' +const USER_UUID = '4bd39765-f83f-4a6f-bfc4-81b88f6ac446' +const ASSISTANT_UUID = '1e54d1be-9919-4b2a-97e2-3292ba55ce0e' +const PROMPT_TEXT = 'hello from the overlap window' +const RESPONSE_TEXT = 'hi from both producers' + +// --------------------------------------------------------------------- +// The proxy producer's half: a wire exchange projected through the claude +// projector inside the gateway dispatcher, with the transcript on disk +// supplying native DAG identity - the exact path a still-proxying session +// takes during the overlap. +// --------------------------------------------------------------------- + +async function proxyRowsForSession() { + const homeDir = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-overlap-')) + try { + const projectsDir = path.join(homeDir, '.claude', 'projects', 'some-repo') + await fs.mkdir(projectsDir, { recursive: true }) + await fs.writeFile( + path.join(projectsDir, `${SESSION}.jsonl`), + [ + JSON.stringify({ + sessionId: SESSION, + uuid: USER_UUID, + parentUuid: null, + type: 'user', + message: { role: 'user', content: PROMPT_TEXT }, + timestamp: '2026-08-17T10:00:00.000Z', + }), + JSON.stringify({ + sessionId: SESSION, + uuid: ASSISTANT_UUID, + parentUuid: USER_UUID, + type: 'assistant', + message: { + role: 'assistant', + id: 'msg_overlap', + content: [{ type: 'text', text: RESPONSE_TEXT }], + }, + timestamp: '2026-08-17T10:00:01.000Z', + }), + ].join('\n') + '\n', + 'utf8' + ) + + const projector = createClaudeExchangeProjector({ + homeDir, + stateFile: path.join(homeDir, 'session-context.jsonl'), + }) + const dispatcher = createAiGatewayMessageProjector({ + gatewayId: 'gw-test', + projectors: [{ ...projector, _seq: 0 }], + }) + return await dispatcher.projectExchange({ + exchange_id: 'ex-overlap', + ts_start: '2026-08-17T10:00:05.000Z', + ts_end: '2026-08-17T10:00:05.250Z', + duration_ms: 250, + upstream: 'anthropic', + provider: null, + method: 'POST', + path: '/v1/messages', + status_code: 200, + request_bytes: 100, + response_bytes: 200, + is_sse: false, + stream_event_count: 0, + request_headers: JSON.stringify({ + 'anthropic-version': '2023-06-01', + 'user-agent': 'claude-cli/1.0', + }), + request_body: JSON.stringify({ + model: 'claude-3-opus', + metadata: { user_id: JSON.stringify({ session_id: SESSION }) }, + messages: [{ role: 'user', content: PROMPT_TEXT }], + }), + response_headers: JSON.stringify({ 'content-type': 'application/json' }), + response_body: JSON.stringify({ + id: 'msg_overlap', + role: 'assistant', + content: [{ type: 'text', text: RESPONSE_TEXT }], + stop_reason: 'end_turn', + }), + error: null, + metadata: JSON.stringify({ dev_run_id: 'run-overlap' }), + stream_events: [], + }) + } finally { + await fs.rm(homeDir, { recursive: true, force: true }) + } +} + +// --------------------------------------------------------------------- +// The OTEL producer's half: the same session as Claude Code's own event +// stream, carrying the same native uuids as `message.uuid`. +// --------------------------------------------------------------------- + +/** @param {Record} attrs */ +function kvAttributes(attrs) { + return Object.entries(attrs).map(([key, value]) => ({ + key, + value: { stringValue: String(value) }, + })) +} + +/** + * @param {string} name + * @param {Record} attrs + * @param {string} timestamp + */ +function record(name, attrs, timestamp) { + return { + timeUnixNano: String(BigInt(Date.parse(timestamp)) * 1_000_000n), + body: { stringValue: `claude_code.${name}` }, + attributes: kvAttributes({ + 'session.id': SESSION, + 'app.version': '2.1.233', + 'app.entrypoint': 'cli', + 'event.name': name, + 'event.timestamp': timestamp, + ...attrs, + }), + } +} + +function otelProjectionForSession() { + const envelope = { + resourceLogs: [ + { + resource: { attributes: kvAttributes({ 'service.name': 'claude-code' }) }, + scopeLogs: [ + { + scope: { name: 'com.anthropic.claude_code.events', version: '2.1.233' }, + logRecords: [ + record('user_prompt', { + prompt: PROMPT_TEXT, + 'message.uuid': USER_UUID, + }, '2026-08-17T10:00:00.100Z'), + record('assistant_response', { + response: RESPONSE_TEXT, + request_id: 'req_overlap', + 'message.uuid': ASSISTANT_UUID, + model: 'claude-3-opus', + }, '2026-08-17T10:00:01.100Z'), + ], + }, + ], + }, + ], + } + const projections = projectClaudeTelemetryEvents(flattenClaudeTelemetryEvents(envelope), { + clientName: 'claude', + usageByRequestId: new Map(), + }) + assert.equal(projections.length, 1) + return projections[0] +} + +/** + * Storage stub with the read surface the pre-write dedupe feature-detects, + * seeded with already-committed part_ids (the proxy producer's rows). + * + * @param {{ committed?: string[] }} [seed] + */ +function makeStorage(seed = {}) { + const committed = seed.committed ?? [] + /** @type {Record[]} */ + const appended = [] + return { + appended, + /** @param {string} dataset @param {string[]} labels */ + cacheTablePath: (dataset, labels) => `/cache/${dataset}/${labels.join('/')}`, + /** @param {{ datasets: string[] }} _scope */ + async discoverCachePartitions(_scope) { + return [{ path: '/cache/committed', partition: {}, rowCount: committed.length }] + }, + async *readRows() { + for (const partId of committed) yield { part_id: partId } + }, + async *readSpooledRows() {}, + /** @param {string} _tablePath @param {unknown} _columns @param {Record[]} rows */ + async appendRows(_tablePath, _columns, rows) { + appended.push(...rows) + }, + } +} + +test('the proxy rows and the OTEL rows for one session share part identity', async () => { + const proxyRows = await proxyRowsForSession() + const otelRows = aiGatewayRowsFromProjectedExchange(otelProjectionForSession()) + + assert.equal(proxyRows.length, 2) + assert.deepEqual( + proxyRows.map((r) => r.part_id).sort(), + otelRows.map((r) => r.part_id).sort() + ) + assert.deepEqual( + proxyRows.map((r) => r.part_id).sort(), + [`${ASSISTANT_UUID}#0`, `${USER_UUID}#0`] + ) + for (const rows of [proxyRows, otelRows]) { + for (const row of rows) assert.equal(row.session_id, SESSION) + } +}) + +test('a session the proxy already stored lands once: the OTEL producer writes nothing new', async () => { + const proxyRows = await proxyRowsForSession() + const storage = makeStorage({ + committed: proxyRows.map((r) => /** @type {string} */ (r.part_id)), + }) + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + + const result = await api.recordProjectedExchange(/** @type {any} */ (otelProjectionForSession())) + assert.deepEqual(result, { rowsWritten: 0, rowsSkipped: 2 }) + assert.equal(storage.appended.length, 0) +}) + +test('a half-stored overlap fills only the gap', async () => { + const proxyRows = await proxyRowsForSession() + const userPartId = /** @type {string} */ ( + proxyRows.find((r) => r.role === 'user')?.part_id + ) + const storage = makeStorage({ committed: [userPartId] }) + const api = createAiGatewayApi(createGatewayState(), { storage: /** @type {any} */ (storage) }) + + const result = await api.recordProjectedExchange(/** @type {any} */ (otelProjectionForSession())) + assert.deepEqual(result, { rowsWritten: 1, rowsSkipped: 1 }) + assert.deepEqual(storage.appended.map((r) => r.part_id), [`${ASSISTANT_UUID}#0`]) +}) diff --git a/test/plugins/claude-settings-attach.test.js b/test/plugins/claude-settings-attach.test.js index 50c9eba5..4761876d 100644 --- a/test/plugins/claude-settings-attach.test.js +++ b/test/plugins/claude-settings-attach.test.js @@ -452,6 +452,7 @@ test('activate() attach() rethrows the JSONC refusal with the refusal mark intac skills: { register() {} }, agents: { register() {} }, initPresets: { register() {} }, + query: { registerDataset() {} }, }) await activateClaude(ctx) diff --git a/test/plugins/claude-settings-otel-attach.test.js b/test/plugins/claude-settings-otel-attach.test.js new file mode 100644 index 00000000..ef4029f6 --- /dev/null +++ b/test/plugins/claude-settings-otel-attach.test.js @@ -0,0 +1,398 @@ +// @ts-check + +import assert from 'node:assert/strict' +import fsp from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { + MODE_BASE_URL, + MODE_OTEL, + MODE_PROXY, + attach, + otelModeEnv, +} from '../../hypaware-core/plugins-workspace/claude/src/settings.js' +import { detachClientFromDisk } from '../../src/core/config/client_detach_disk.js' +import { ensureLocalCa } from '../../src/core/tls/ca.js' + +const PORT = 18521 +const TELEMETRY_PORT = 4319 + +/** + * A temp home with an optional seeded settings file. + * + * @param {Record} [settings] + */ +async function rig(settings) { + const root = await fsp.mkdtemp(path.join(os.tmpdir(), 'hyp-otel-attach-')) + const settingsPath = path.join(root, '.claude', 'settings.json') + await fsp.mkdir(path.dirname(settingsPath), { recursive: true }) + if (settings) await fsp.writeFile(settingsPath, JSON.stringify(settings, null, 2) + '\n') + + return { + root, + stateRoot: path.join(root, '.hyp', 'hypaware'), + settingsPath, + stateFile: path.join(root, 'session-context.jsonl'), + spoolDir: path.join(root, '.hyp', 'spool', 'claude-bodies'), + /** @returns {Promise>} */ + async read() { + return JSON.parse(await fsp.readFile(settingsPath, 'utf8')) + }, + /** @returns {Promise} */ + async raw() { + return fsp.readFile(settingsPath, 'utf8') + }, + /** The env a core detach resolves the settings file from. */ + env: { HOME: root, HYP_HOME: path.join(root, '.hyp') }, + cleanup: () => fsp.rm(root, { recursive: true, force: true }), + } +} + +/** @param {{ settingsPath: string, stateFile: string, spoolDir: string }} r */ +function otelAttach(r, extra = {}) { + return attach({ + port: PORT, + version: '2.0.0', + stateFile: r.stateFile, + settingsPath: r.settingsPath, + mode: MODE_OTEL, + telemetryPort: TELEMETRY_PORT, + spoolDir: r.spoolDir, + claudeVersion: '2.1.233', + ...extra, + }) +} + +/** The claude descriptor, as the core undo receives it. */ +const CLAUDE_DESCRIPTOR = { + name: 'claude', + plugin: '@hypaware/claude', + attachProbe: { + format: 'json', + settings_file: '.claude/settings.json', + marker_key: '_hypaware', + }, +} + +// The list IS the decision: the golden compare pins every key by value, so a +// silently renamed or dropped flag fails here rather than as an empty dataset. +// @ref LLP 0258#env-keys [tests]: exactly these keys, with exactly these values +test('otel attach writes exactly the telemetry env block', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + + await otelAttach(r) + const value = await r.read() + + assert.equal(value.env.CLAUDE_CODE_ENABLE_TELEMETRY, '1') + assert.equal(value.env.OTEL_LOGS_EXPORTER, 'otlp') + assert.equal(value.env.OTEL_METRICS_EXPORTER, 'otlp') + assert.equal(value.env.OTEL_EXPORTER_OTLP_PROTOCOL, 'http/json') + assert.equal(value.env.OTEL_EXPORTER_OTLP_ENDPOINT, `http://127.0.0.1:${TELEMETRY_PORT}`) + assert.equal(value.env.OTEL_LOG_USER_PROMPTS, '1') + assert.equal(value.env.OTEL_LOG_ASSISTANT_RESPONSES, '1') + assert.equal(value.env.OTEL_LOG_TOOL_DETAILS, '1') + assert.equal(value.env.OTEL_LOG_RAW_API_BODIES, `file:${r.spoolDir}`) + + // The marker manages that block and nothing else, so the core undo removes + // exactly what attach added. + const expectedManaged = Object.fromEntries( + otelModeEnv({ telemetryPort: TELEMETRY_PORT, spoolDir: r.spoolDir }) + .map(({ key, value: v }) => [key, v]) + ) + assert.deepEqual(value._hypaware.managed.env, expectedManaged) +}) + +// The Remote Control predicate, stated as absences: nothing this mode writes +// routes traffic, so the endpoint stays first party with no override keys. +// @ref LLP 0258#env-keys [tests]: no base URL, no proxy keys, no first-party overrides +test('otel attach writes no routing key and no first-party override', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + + await otelAttach(r) + const value = await r.read() + + assert.equal(Object.hasOwn(value.env, 'ANTHROPIC_BASE_URL'), false) + assert.equal(Object.hasOwn(value.env, 'HTTPS_PROXY'), false) + assert.equal(Object.hasOwn(value.env, 'NODE_EXTRA_CA_CERTS'), false) + assert.equal(Object.hasOwn(value.env, 'ENABLE_TOOL_SEARCH'), false) + assert.equal(Object.hasOwn(value.env, '_CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL'), false) +}) + +// @ref LLP 0258#marker-and-spool [tests]: the marker records the mode and the spool directory +test('otel attach records mode and spool directory on the marker', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + + await otelAttach(r) + const value = await r.read() + + assert.equal(value._hypaware.mode, 'otel') + assert.equal(value._hypaware.spool_dir, r.spoolDir) + // `port` stays the gateway's: it is what the attach-drift check compares. + assert.equal(value._hypaware.port, PORT) +}) + +test('otel attach still installs the managed session hooks', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + + await otelAttach(r) + const value = await r.read() + + const events = Object.keys(value.hooks).sort() + assert.deepEqual(events, ['CwdChanged', 'PostToolUse', 'SessionStart', 'UserPromptSubmit']) + assert.match(JSON.stringify(value.hooks), /claude-hook session-context/) + assert.match(JSON.stringify(value.hooks), /claude-hook classify-cwd/) +}) + +// Below the floor the client emits none of the events the listener reads, so +// attach refuses rather than writing a settings file that says "attached" +// over a capture that never starts. No fallback to any other mode. +// @ref LLP 0258#version-floor [tests]: refusal, with the upgrade hint, before any settings I/O +test('otel attach refuses below the version floor and leaves settings untouched', async (t) => { + const r = await rig({ env: { ANTHROPIC_API_KEY: 'sk-user-key' } }) + t.after(() => r.cleanup()) + + const before = await r.raw() + await assert.rejects( + () => otelAttach(r, { claudeVersion: '2.1.192' }), + (err) => { + assert.equal(/** @type {any} */ (err).code, 'VERSION_FLOOR') + assert.match(String(/** @type {any} */ (err).message), /claude update/) + return true + } + ) + assert.equal(await r.raw(), before) +}) + +// "Leaves any existing attach untouched" includes the mode being switched +// away from: a proxy-attached machine on an old client keeps its working +// proxy attach byte for byte. +// @ref LLP 0258#version-floor [tests]: an existing attach survives the refusal +test('a floor refusal leaves an existing proxy attach in place', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + const ca = await ensureLocalCa({ stateRoot: r.stateRoot, hosts: ['api.anthropic.com'] }) + + await attach({ + port: PORT, + version: '2.0.0', + stateFile: r.stateFile, + settingsPath: r.settingsPath, + mode: MODE_PROXY, + caCertPath: ca.certPath, + }) + const before = await r.raw() + + await assert.rejects( + () => otelAttach(r, { claudeVersion: '2.0.0' }), + /claude update/ + ) + assert.equal(await r.raw(), before) + assert.equal((await r.read())._hypaware.mode, 'proxy') +}) + +// Unknown is not old: refusing on "we could not tell" would block exactly the +// machines most likely to be current. +test('an undetectable version attaches', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + + await otelAttach(r, { claudeVersion: undefined }) + assert.equal((await r.read())._hypaware.mode, 'otel') +}) + +test('the floor itself attaches (2.1.193 is not below 2.1.193)', async (t) => { + const r = await rig({}) + t.after(() => r.cleanup()) + + await otelAttach(r, { claudeVersion: '2.1.193' }) + assert.equal((await r.read())._hypaware.mode, 'otel') +}) + +test('otel attach requires the telemetry port and an absolute spool path', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + await assert.rejects( + () => otelAttach(r, { telemetryPort: undefined }), + (err) => /** @type {any} */ (err).code === 'INVALID_TELEMETRY_PORT' + ) + await assert.rejects( + () => otelAttach(r, { spoolDir: undefined }), + (err) => /** @type {any} */ (err).code === 'INVALID_SPOOL_DIR' + ) + await assert.rejects( + () => otelAttach(r, { spoolDir: 'relative/spool' }), + (err) => /** @type {any} */ (err).code === 'INVALID_SPOOL_DIR' + ) + // Nothing was written by any refused validation. + await assert.rejects(() => r.raw(), (err) => /** @type {any} */ (err).code === 'ENOENT') +}) + +// A pre-existing OTEL endpoint is almost always the user's own collector: +// taken over with a backup and a notice, never silently, and the notice never +// echoes the value (collector endpoints carry tokens). +// @ref LLP 0044#conflict-back-up--override-restore-on-leave [tests] +test('an existing OTEL endpoint is backed up, warned about without the value, and restored', async (t) => { + const r = await rig({ + env: { OTEL_EXPORTER_OTLP_ENDPOINT: 'https://token@collector.corp:4318' }, + }) + t.after(() => r.cleanup()) + + const result = await otelAttach(r) + assert.equal(result.changed, true) + const warned = String(result.changed && result.warnings?.join(' ')) + assert.match(warned, /OTEL_EXPORTER_OTLP_ENDPOINT/) + assert.doesNotMatch(warned, /collector\.corp/) + // The display copy is redacted; the marker's backup is verbatim. + assert.equal(result.changed && result.prevValue, 'https://***@collector.corp:4318') + + const attached = await r.read() + assert.equal( + attached._hypaware.prev_env.OTEL_EXPORTER_OTLP_ENDPOINT, + 'https://token@collector.corp:4318' + ) + + await detachClientFromDisk({ + descriptor: /** @type {never} */ (CLAUDE_DESCRIPTOR), + homeDir: r.root, + env: r.env, + }) + const detached = await r.read() + assert.equal(detached.env.OTEL_EXPORTER_OTLP_ENDPOINT, 'https://token@collector.corp:4318') + assert.equal(Object.hasOwn(detached.env, 'CLAUDE_CODE_ENABLE_TELEMETRY'), false) +}) + +// Migrating an already-attached machine must not strand the old mode's keys. +// @ref LLP 0232#mode-migration [tests]: the same key release, in the new direction +test('switching from base-URL to otel mode releases the old keys', async (t) => { + const r = await rig({ env: { ANTHROPIC_BASE_URL: 'https://gw.corp.example' } }) + t.after(() => r.cleanup()) + + await attach({ + port: PORT, + version: '2.0.0', + stateFile: r.stateFile, + settingsPath: r.settingsPath, + mode: MODE_BASE_URL, + }) + await otelAttach(r) + const value = await r.read() + + // The user's own base URL came back rather than being left pointed at us. + assert.equal(value.env.ANTHROPIC_BASE_URL, 'https://gw.corp.example') + assert.equal(Object.hasOwn(value.env, 'ENABLE_TOOL_SEARCH'), false) + assert.equal(Object.hasOwn(value.env, '_CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL'), false) + assert.equal(value.env.CLAUDE_CODE_ENABLE_TELEMETRY, '1') + assert.equal(value._hypaware.mode, 'otel') +}) + +test('switching from proxy to otel mode releases the proxy keys', async (t) => { + const r = await rig({ env: { HTTPS_PROXY: 'http://proxy.corp:8080' } }) + t.after(() => r.cleanup()) + const ca = await ensureLocalCa({ stateRoot: r.stateRoot, hosts: ['api.anthropic.com'] }) + + await attach({ + port: PORT, + version: '2.0.0', + stateFile: r.stateFile, + settingsPath: r.settingsPath, + mode: MODE_PROXY, + caCertPath: ca.certPath, + }) + await otelAttach(r) + const value = await r.read() + + assert.equal(value.env.HTTPS_PROXY, 'http://proxy.corp:8080') + assert.equal(Object.hasOwn(value.env, 'NODE_EXTRA_CA_CERTS'), false) + assert.equal(value.env.OTEL_EXPORTER_OTLP_ENDPOINT, `http://127.0.0.1:${TELEMETRY_PORT}`) + assert.equal(value._hypaware.mode, 'otel') +}) + +// Detach stays the core disk-driven marker replay: no adapter code, and the +// settings end as if HypAware was never there. +// @ref LLP 0045#part-3-reverse-runs-from-disk-the-marker-is-a-self-describing-undo-record [tests] +test('detach after an otel attach restores the settings byte for byte', async (t) => { + const seed = { + env: { ANTHROPIC_API_KEY: 'sk-user-key' }, + permissions: { allow: ['Bash(ls *)'] }, + } + const r = await rig(seed) + t.after(() => r.cleanup()) + const seedBody = await r.raw() + + await otelAttach(r) + await detachClientFromDisk({ + descriptor: /** @type {never} */ (CLAUDE_DESCRIPTOR), + homeDir: r.root, + env: r.env, + }) + + assert.equal(await r.raw(), seedBody) +}) + +test('a re-attach keeps the original backup rather than backing up our own value', async (t) => { + const r = await rig({ env: { OTEL_EXPORTER_OTLP_ENDPOINT: 'http://own-collector:4318' } }) + t.after(() => r.cleanup()) + + await otelAttach(r) + const second = await otelAttach(r) + + const value = await r.read() + assert.equal(value._hypaware.prev_env.OTEL_EXPORTER_OTLP_ENDPOINT, 'http://own-collector:4318') + // Nothing new was displaced this run, so nothing new is warned about. + assert.equal(second.changed && second.warnings, undefined) +}) + +// A per-signal OTLP key outranks the generic endpoint attach writes, so a +// machine carrying one exports its telemetry - including the prompt and +// response text this attach turns on - to the collector that key names, while +// `hyp status` says `attached (otel)` and the listener sees nothing. Attach +// manages exactly the nine keys LLP 0258 names and no more, so the only honest +// answer is to say so out loud. +// @ref LLP 0258#env-keys [tests]: the managed set is unchanged; what is outside it and outranks it is named +test('a per-signal OTLP override is warned about, without echoing its value', async (t) => { + const r = await rig({ + env: { + OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: 'https://token@collector.corp:4318', + OTEL_EXPORTER_OTLP_HEADERS: 'authorization=Bearer sekrit', + }, + }) + t.after(() => r.cleanup()) + + const result = await otelAttach(r) + assert.equal(result.changed, true) + const warned = String(result.changed && result.warnings?.join(' ')) + assert.match(warned, /OTEL_EXPORTER_OTLP_LOGS_ENDPOINT/) + assert.match(warned, /OTEL_EXPORTER_OTLP_HEADERS/) + assert.match(warned, /outranks/) + // Neither the collector nor the credential appears: this string is printed, + // logged, and serialised into `--json`. + assert.doesNotMatch(warned, /collector\.corp/) + assert.doesNotMatch(warned, /sekrit/) + + // Warned about, not touched: they are outside the managed set, so attach + // leaves them exactly as it found them and detach has nothing to restore. + const attached = await r.read() + assert.equal( + attached.env.OTEL_EXPORTER_OTLP_LOGS_ENDPOINT, + 'https://token@collector.corp:4318' + ) + assert.equal( + Object.hasOwn(attached._hypaware.managed.env, 'OTEL_EXPORTER_OTLP_LOGS_ENDPOINT'), + false + ) +}) + +test('an ordinary otel attach warns about no per-signal key', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + const result = await otelAttach(r) + assert.equal(result.changed, true) + assert.equal(result.changed && result.warnings, undefined) +}) diff --git a/test/plugins/claude-telemetry-attach-port.test.js b/test/plugins/claude-telemetry-attach-port.test.js new file mode 100644 index 00000000..680b26e1 --- /dev/null +++ b/test/plugins/claude-telemetry-attach-port.test.js @@ -0,0 +1,101 @@ +// @ts-check + +import assert from 'node:assert/strict' +import fsp from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { + DEFAULT_TELEMETRY_PORT, + resolveAttachTelemetryPort, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/source.js' +import { writePidFile } from '../../src/core/daemon/pid.js' +import { writeStatusFile } from '../../src/core/daemon/status.js' + +async function rig() { + const root = await fsp.mkdtemp(path.join(os.tmpdir(), 'hyp-attach-port-')) + return { + stateRoot: path.join(root, 'hypaware'), + cleanup: () => fsp.rm(root, { recursive: true, force: true }), + } +} + +/** @param {string} stateRoot @param {number} pid @param {number} listenPort */ +function writeLiveStatus(stateRoot, pid, listenPort) { + writePidFile(stateRoot, { + pid, + startedAt: new Date().toISOString(), + runId: 'test-run', + mode: 'foreground', + }) + writeStatusFile(stateRoot, /** @type {any} */ ({ + sources: [ + { name: 'ai-gateway', plugin: '@hypaware/ai-gateway', details: { host: '127.0.0.1', port: 18521 } }, + { name: 'claude-telemetry', plugin: '@hypaware/claude', details: { listen_host: '127.0.0.1', listen_port: listenPort } }, + ], + })) +} + +test('no daemon and no config resolves the well-known default', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + const port = resolveAttachTelemetryPort({ stateRoot: r.stateRoot, config: {} }) + assert.equal(port, DEFAULT_TELEMETRY_PORT) +}) + +test('a configured fixed port wins over the default', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + const port = resolveAttachTelemetryPort({ + stateRoot: r.stateRoot, + config: { telemetry: { listen_port: 5555 } }, + }) + assert.equal(port, 5555) +}) + +// A configured 0 asks for a dynamic port, which no attach can know until a +// daemon publishes the bound one; it must not leak into the endpoint. +test('a configured dynamic port (0) reads as unconfigured', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + const port = resolveAttachTelemetryPort({ + stateRoot: r.stateRoot, + config: { telemetry: { listen_port: 0 } }, + }) + assert.equal(port, DEFAULT_TELEMETRY_PORT) +}) + +// The bind fallback moves the listener off its default when the port is +// taken; the promise that makes that safe is attach reading the bound port +// back off the source status. +// @ref LLP 0114#explicit-listen-fails-loudly [tests]: the fallback is only safe because attach reads the real bound port +test('a live daemon status wins over config and default', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + writeLiveStatus(r.stateRoot, process.pid, 6666) + const port = resolveAttachTelemetryPort({ + stateRoot: r.stateRoot, + config: { telemetry: { listen_port: 5555 } }, + }) + assert.equal(port, 6666) +}) + +// A status.json outlives its daemon; a dead pid must not hand back a port +// nobody is listening on when the config still names a real one. +test('a dead daemon status is ignored', async (t) => { + const r = await rig() + t.after(() => r.cleanup()) + + // Far above any real pid ceiling on macOS/Linux, so signal 0 fails. + writeLiveStatus(r.stateRoot, 2147483646, 6666) + const port = resolveAttachTelemetryPort({ + stateRoot: r.stateRoot, + config: { telemetry: { listen_port: 5555 } }, + }) + assert.equal(port, 5555) +}) diff --git a/test/plugins/claude-telemetry-bodies.test.js b/test/plugins/claude-telemetry-bodies.test.js new file mode 100644 index 00000000..a4e97710 --- /dev/null +++ b/test/plugins/claude-telemetry-bodies.test.js @@ -0,0 +1,385 @@ +// @ts-check + +/** + * The body half of the Claude telemetry listener: reading spooled body + * files, refusing refs that point outside the spool, and joining a + * body's gap blocks (untruncated tool args, thinking signatures, tool + * results) into the projection the events alone cannot complete. + * + * @ref LLP 0257#testing [tests]: event-plus-body projection identity is unit + * tested in the root suite; the end-to-end seam is the hermetic smoke + */ + +import assert from 'node:assert/strict' +import fsp from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { + BODY_EVENT_NAMES, + deleteSpooledBodies, + deleteSpooledBodiesForEvents, + loadSpooledBodies, + requestBodyFacts, + spooledBodyGapMessages, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/bodies.js' +import { + SESSION_BODY_FACTS_LIMIT, + projectClaudeTelemetryEvents, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/projection.js' +import { aiGatewayRowsFromProjectedExchange } from '../../hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js' + +const SESSION = 'e53c128d-9f45-470f-86f1-d5b5f3766708' +const REQUEST_ID = 'req_011Ce8sjpb8Uzvot2JMvFkKe' +const LONG_ARG = 'x'.repeat(600) + +/** + * @param {string} name + * @param {Record} attrs + * @param {string} [timestamp] + */ +function evt(name, attrs, timestamp = '2026-08-17T19:30:24.450Z') { + return { name, attributes: { 'session.id': SESSION, ...attrs }, timestamp } +} + +function requestBody() { + return { + model: 'claude-haiku-4-5-20251001', + system: [{ type: 'text', text: 'You are a coding agent.' }], + tools: [{ name: 'Read', description: 'Read a file', input_schema: { type: 'object' } }], + messages: [ + { role: 'user', content: 'Run ls, then read notes.txt.' }, + { + role: 'assistant', + content: [ + { type: 'text', text: 'Reading it now.' }, + { type: 'tool_use', id: 'toolu_1', name: 'Read', input: { file_path: '/tmp/notes.txt', pad: LONG_ARG } }, + ], + }, + { + role: 'user', + content: [{ type: 'tool_result', tool_use_id: 'toolu_1', content: 'notes: spike findings' }], + }, + ], + } +} + +/** @param {boolean} withText */ +function responseBody(withText) { + return { + id: 'msg_smoke', + type: 'message', + role: 'assistant', + model: 'claude-haiku-4-5-20251001', + content: [ + { type: 'thinking', thinking: 'It is a spike repo.', signature: 'sig-abc' }, + ...(withText ? [{ type: 'text', text: 'This is a spike repo.' }] : []), + ], + stop_reason: withText ? 'end_turn' : 'tool_use', + usage: { input_tokens: 73, output_tokens: 113 }, + } +} + +async function tmpSpool() { + const root = await fsp.mkdtemp(path.join(os.tmpdir(), 'hyp-bodies-')) + return path.join(root, 'spool', 'claude-bodies') +} + +/** + * @param {string} dir + * @param {string} name + * @param {unknown} body + */ +async function writeBody(dir, name, body) { + await fsp.mkdir(dir, { recursive: true }) + const file = path.join(dir, name) + await fsp.writeFile(file, typeof body === 'string' ? body : JSON.stringify(body)) + return file +} + +test('loadSpooledBodies reads the request and response files a batch references', async () => { + const dir = await tmpSpool() + const reqFile = await writeBody(dir, 'a.request.json', requestBody()) + const respFile = await writeBody(dir, 'a.response.json', responseBody(true)) + const events = [ + evt('api_request_body', { body_ref: reqFile, request_id: REQUEST_ID }), + evt('api_response_body', { body_ref: respFile, request_id: REQUEST_ID }), + ] + const spooled = await loadSpooledBodies(events, { spoolDir: dir }) + assert.equal(spooled.bodies.size, 2) + assert.equal(spooled.bodies.get(reqFile)?.kind, 'request') + assert.equal(spooled.bodies.get(respFile)?.kind, 'response') + assert.deepEqual(spooled.consumedFiles.sort(), [reqFile, respFile].sort()) + assert.ok(spooled.consumedBytes > 0) + assert.equal(spooled.missing, 0) + assert.equal(spooled.unparseable, 0) + assert.deepEqual(spooled.refused, []) +}) + +test('a body_ref outside the spool is refused and its file is left alone', async () => { + const dir = await tmpSpool() + await fsp.mkdir(dir, { recursive: true }) + // A sibling of the spool: contained refs must be under the spool root, + // not merely share its prefix. + const outside = await writeBody(path.dirname(dir), 'secret.json', { private: true }) + const traversal = path.join(dir, '..', 'secret.json') + const spooled = await loadSpooledBodies( + [ + evt('api_request_body', { body_ref: outside }), + evt('api_request_body', { body_ref: traversal }), + ], + { spoolDir: dir } + ) + assert.equal(spooled.bodies.size, 0) + assert.equal(spooled.refused.length, 2) + // Refused means untouched: never read, never deleted. + assert.deepEqual(JSON.parse(await fsp.readFile(outside, 'utf8')), { private: true }) +}) + +test('a missing body file counts as missing, not as an error', async () => { + const dir = await tmpSpool() + await fsp.mkdir(dir, { recursive: true }) + const spooled = await loadSpooledBodies( + [evt('api_request_body', { body_ref: path.join(dir, 'evicted.json') })], + { spoolDir: dir } + ) + assert.equal(spooled.bodies.size, 0) + assert.equal(spooled.missing, 1) +}) + +test('an unparseable body is deleted immediately and counted', async () => { + const dir = await tmpSpool() + const file = await writeBody(dir, 'broken.json', 'not json {') + const spooled = await loadSpooledBodies( + [evt('api_request_body', { body_ref: file })], + { spoolDir: dir } + ) + assert.equal(spooled.bodies.size, 0) + assert.equal(spooled.unparseable, 1) + await assert.rejects(fsp.stat(file)) +}) + +test('deleteSpooledBodies removes projected files and tolerates absence', async () => { + const dir = await tmpSpool() + const file = await writeBody(dir, 'done.json', {}) + const deleted = await deleteSpooledBodies([file, path.join(dir, 'never-existed.json')]) + assert.equal(deleted, 2) + await assert.rejects(fsp.stat(file)) +}) + +// @ref LLP 0253#delete-on-drop [tests] / LLP 0256#bodies-deleted [tests]: a +// policy-dropped session's bodies are deleted unread, under the same +// spool-containment rule as the read path. +test('deleteSpooledBodiesForEvents removes a dropped session\'s bodies without reading them', async () => { + const dir = await tmpSpool() + const reqFile = await writeBody(dir, 'dropped.request.json', requestBody()) + const respFile = await writeBody(dir, 'dropped.response.json', responseBody(true)) + const events = [ + evt('user_prompt', { prompt: 'secret' }), + evt('api_request_body', { body_ref: reqFile, request_id: REQUEST_ID }), + // The same ref twice must not double-count. + evt('api_request_body', { body_ref: reqFile, request_id: REQUEST_ID }), + evt('api_response_body', { body_ref: respFile, request_id: REQUEST_ID }), + ] + const removal = await deleteSpooledBodiesForEvents(events, { spoolDir: dir }) + assert.equal(removal.deleted, 2) + assert.deepEqual(removal.refused, []) + await assert.rejects(fsp.stat(reqFile)) + await assert.rejects(fsp.stat(respFile)) +}) + +test('deleteSpooledBodiesForEvents refuses refs outside the spool and leaves them alone', async () => { + const dir = await tmpSpool() + await fsp.mkdir(dir, { recursive: true }) + // The ref arrives over the wire from whatever found the loopback port, so + // an uncontained ref would turn the DROP into a delete primitive over the + // whole filesystem, same threat as the read path's. + const outside = await writeBody(path.dirname(dir), 'not-ours.json', { private: true }) + // Concatenated, not path.join'd: join would normalize the `..` away and + // the two refs would be one string, which the ref-level dedupe collapses. + const traversal = dir + path.sep + '..' + path.sep + 'not-ours.json' + const removal = await deleteSpooledBodiesForEvents( + [ + evt('api_request_body', { body_ref: outside }), + evt('api_response_body', { body_ref: traversal }), + ], + { spoolDir: dir } + ) + assert.equal(removal.deleted, 0) + assert.equal(removal.refused.length, 2) + assert.deepEqual(JSON.parse(await fsp.readFile(outside, 'utf8')), { private: true }) +}) + +test('requestBodyFacts pulls system, tools, and model from a request body only', () => { + const facts = requestBodyFacts({ kind: 'request', file: '/s/a.json', body: requestBody() }) + assert.equal(facts.system_text, 'You are a coding agent.') + assert.equal(facts.model, 'claude-haiku-4-5-20251001') + assert.equal(/** @type {any} */ (facts.tools)?.[0]?.name, 'Read') + assert.deepEqual( + requestBodyFacts({ kind: 'response', file: '/s/b.json', body: responseBody(true) }), + {} + ) +}) + +test('body gap blocks join the projection in body order, untruncated', () => { + const reqFile = '/spool/a.request.json' + const respFile = '/spool/a.response.json' + const events = [ + evt('user_prompt', { prompt: 'Run ls, then read notes.txt.', 'message.uuid': 'u-user' }), + evt('api_request_body', { body_ref: reqFile, request_id: REQUEST_ID }, '2026-08-17T19:30:26.000Z'), + evt('api_request', { request_id: REQUEST_ID, output_tokens: 113 }, '2026-08-17T19:30:31.009Z'), + evt('api_response_body', { body_ref: respFile, request_id: REQUEST_ID }, '2026-08-17T19:30:31.009Z'), + evt('assistant_response', { response: 'This is a spike repo.', request_id: REQUEST_ID, 'message.uuid': 'u-asst' }, '2026-08-17T19:30:31.009Z'), + ] + const spooledBodies = new Map([ + [reqFile, /** @type {any} */ ({ kind: 'request', file: reqFile, body: requestBody() })], + [respFile, /** @type {any} */ ({ kind: 'response', file: respFile, body: responseBody(true) })], + ]) + const [projection] = projectClaudeTelemetryEvents(events, { + clientName: 'claude', + usageByRequestId: new Map(), + spooledBodies, + }) + + // Exchange-level: the fields only the request body carries. + assert.equal(projection.system_text, 'You are a coding agent.') + assert.equal(/** @type {any} */ (projection.tools)?.[0]?.name, 'Read') + assert.equal(projection.model, 'claude-haiku-4-5-20251001') + + // Message order follows the stream, with each body's gap blocks at the + // body event's position: prompt, tool_use, tool_result, thinking, text. + // The bodies' plain text blocks are NOT re-projected. + const kinds = projection.messages.map((m) => + typeof m.content === 'string' ? 'text' : /** @type {any} */ (m.content[0]).type + ) + assert.deepEqual(kinds, ['text', 'tool_use', 'tool_result', 'thinking', 'text']) + + const toolUse = /** @type {any} */ (projection.messages[1]) + assert.equal(toolUse.role, 'assistant') + assert.equal(toolUse.content[0].input.pad, LONG_ARG) + assert.equal(toolUse.raw_frame?.type, 'api_request_body') + assert.equal(toolUse.raw_frame?.body_file, 'a.request.json') + + const toolResult = /** @type {any} */ (projection.messages[2]) + assert.equal(toolResult.role, 'user') + assert.equal(toolResult.content[0].tool_use_id, 'toolu_1') + + const thinking = /** @type {any} */ (projection.messages[3]) + assert.equal(thinking.content[0].signature, 'sig-abc') + assert.equal(thinking.raw_frame?.type, 'api_response_body') + assert.equal(thinking.raw_frame?.message_id, 'msg_smoke') + + // The minimized frame carries pointers, never content. + for (const message of [toolUse, toolResult, thinking]) { + const frame = JSON.stringify(message.raw_frame) + assert.ok(!frame.includes(LONG_ARG.slice(0, 32))) + assert.ok(!frame.includes('spike')) + } +}) + +test('a response body with no text block claims the usage its event row never gets', () => { + const respFile = '/spool/b.response.json' + const events = [ + evt('api_request', { request_id: REQUEST_ID, output_tokens: 113 }), + evt('api_response_body', { body_ref: respFile, request_id: REQUEST_ID }), + ] + const usage = new Map() + const [projection] = projectClaudeTelemetryEvents(events, { + clientName: 'claude', + usageByRequestId: usage, + spooledBodies: new Map([ + [respFile, /** @type {any} */ ({ kind: 'response', file: respFile, body: responseBody(false) })], + ]), + }) + const [thinking] = /** @type {any[]} */ (projection.messages) + assert.equal(thinking.stop_reason, 'tool_use') + assert.equal(thinking.attributes?.usage?.output_tokens, 113) + // Claimed once, so a later batch cannot double-count it. + assert.equal(usage.size, 0) +}) + +test('a response body with a text block leaves usage to the assistant_response event', () => { + const respFile = '/spool/c.response.json' + const events = [ + evt('api_request', { request_id: REQUEST_ID, output_tokens: 113 }), + evt('api_response_body', { body_ref: respFile, request_id: REQUEST_ID }), + evt('assistant_response', { response: 'This is a spike repo.', request_id: REQUEST_ID, 'message.uuid': 'u-asst' }), + ] + const [projection] = projectClaudeTelemetryEvents(events, { + clientName: 'claude', + usageByRequestId: new Map(), + spooledBodies: new Map([ + [respFile, /** @type {any} */ ({ kind: 'response', file: respFile, body: responseBody(true) })], + ]), + }) + const thinking = /** @type {any} */ (projection.messages[0]) + const text = /** @type {any} */ (projection.messages[1]) + assert.equal(thinking.attributes?.usage, undefined) + assert.equal(text.attributes?.usage?.output_tokens, 113) +}) + +test('body-derived rows expand to the same part ids on replay', () => { + const reqFile = '/spool/a.request.json' + const project = () => projectClaudeTelemetryEvents( + [evt('api_request_body', { body_ref: reqFile, request_id: REQUEST_ID })], + { + clientName: 'claude', + usageByRequestId: new Map(), + spooledBodies: new Map([ + [reqFile, /** @type {any} */ ({ kind: 'request', file: reqFile, body: requestBody() })], + ]), + } + ) + const first = aiGatewayRowsFromProjectedExchange(project()[0]) + const second = aiGatewayRowsFromProjectedExchange(project()[0]) + assert.ok(first.length > 0) + assert.deepEqual(first.map((r) => r.part_id), second.map((r) => r.part_id)) +}) + +test('session body facts carry to a later batch of the same session, bounded', () => { + const reqFile = '/spool/a.request.json' + const sessionBodyFacts = new Map() + const spooledBodies = new Map([ + [reqFile, /** @type {any} */ ({ kind: 'request', file: reqFile, body: requestBody() })], + ]) + projectClaudeTelemetryEvents( + [evt('api_request_body', { body_ref: reqFile, request_id: REQUEST_ID })], + { clientName: 'claude', usageByRequestId: new Map(), spooledBodies, sessionBodyFacts } + ) + // A later batch: same session, no body event at all (the exporter split + // the turn), still stamps the remembered system prompt and tools. + const [later] = projectClaudeTelemetryEvents( + [evt('assistant_response', { response: 'Later turn.', 'message.uuid': 'u-later' })], + { clientName: 'claude', usageByRequestId: new Map(), sessionBodyFacts } + ) + assert.equal(later.system_text, 'You are a coding agent.') + assert.equal(/** @type {any} */ (later.tools)?.[0]?.name, 'Read') + + // The carry-over map is bounded, oldest session evicted first: flood it + // with fresh sessions and the original session's facts fall out. + const floodEvents = [] + const floodBodies = new Map() + for (let i = 0; i < SESSION_BODY_FACTS_LIMIT + 3; i++) { + const file = `/spool/flood-${i}.request.json` + floodBodies.set(file, { kind: 'request', file, body: requestBody() }) + floodEvents.push({ + name: 'api_request_body', + attributes: { 'session.id': `flood-${i}`, body_ref: file }, + timestamp: '2026-08-17T19:31:00.000Z', + }) + } + projectClaudeTelemetryEvents(floodEvents, { + clientName: 'claude', + usageByRequestId: new Map(), + spooledBodies: floodBodies, + sessionBodyFacts, + }) + assert.equal(sessionBodyFacts.size, SESSION_BODY_FACTS_LIMIT) + assert.equal(sessionBodyFacts.has(SESSION), false) + assert.equal(sessionBodyFacts.has(`flood-${SESSION_BODY_FACTS_LIMIT + 2}`), true) +}) + +test('BODY_EVENT_NAMES names exactly the two body events', () => { + assert.deepEqual([...BODY_EVENT_NAMES], ['api_request_body', 'api_response_body']) +}) diff --git a/test/plugins/claude-telemetry-events-dataset.test.js b/test/plugins/claude-telemetry-events-dataset.test.js new file mode 100644 index 00000000..114f4446 --- /dev/null +++ b/test/plugins/claude-telemetry-events-dataset.test.js @@ -0,0 +1,312 @@ +// @ts-check + +/** + * The `claude_telemetry_events` dataset: the row split (behavioral + * events in, content and body events out; hot fields typed, the rest in + * the attributes JSON), the metrics decoder that feeds it, and the + * registration's cache roundtrip. + * + * @ref LLP 0255#row-shape [tests]: one row per event, typed hot fields, + * no attribute dropped + */ + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { createQueryStorageService } from '../../src/core/cache/storage.js' +import { + flattenClaudeTelemetryMetrics, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/events.js' +import { + CLAUDE_TELEMETRY_EVENT_COLUMNS, + TELEMETRY_EVENTS_DATASET, + claudeTelemetryDatasetRegistration, + claudeTelemetryEventRows, + claudeTelemetryTablePath, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/events_dataset.js' + +/** + * @import { ClaudeTelemetryEvent } from '../../hypaware-core/plugins-workspace/claude/src/types.js' + */ + +const SESSION = 'e53c128d-9f45-470f-86f1-d5b5f3766708' + +/** + * @param {string} name + * @param {Record} attrs + * @returns {ClaudeTelemetryEvent} + */ +function event(name, attrs = {}) { + return { + name, + timestamp: '2026-08-17T19:30:20.000Z', + attributes: { + 'event.name': name, + 'event.timestamp': '2026-08-17T19:30:20.000Z', + 'session.id': SESSION, + 'app.version': '2.1.233', + ...attrs, + }, + } +} + +// --------------------------------------------------------------------- +// claudeTelemetryEventRows: the behavioral/content split and the row shape +// --------------------------------------------------------------------- + +test('content and body-pointer events yield no behavioral rows', () => { + const rows = claudeTelemetryEventRows([ + event('user_prompt', { prompt: 'secret prompt text', 'message.uuid': 'u-1' }), + event('assistant_response', { response: 'secret response', 'message.uuid': 'u-2' }), + event('api_request_body', { body_ref: '/tmp/spool/req.json' }), + event('api_response_body', { body_ref: '/tmp/spool/resp.json' }), + ]) + assert.deepEqual(rows, []) +}) + +test('a tool_decision event becomes one row with the hot fields typed and lifted out of the JSON', () => { + const rows = claudeTelemetryEventRows([ + event('tool_decision', { + tool_name: 'Read', + decision: 'reject', + source: 'user_reject', + language: 'javascript', + }), + ]) + assert.equal(rows.length, 1) + const row = rows[0] + assert.equal(row.event_name, 'tool_decision') + assert.equal(row.event_timestamp, '2026-08-17T19:30:20.000Z') + assert.equal(row.session_id, SESSION) + assert.equal(row.tool_name, 'Read') + assert.equal(row.decision, 'reject') + assert.equal(row.source, 'user_reject') + assert.equal(row.cost_usd, null) + const attrs = /** @type {Record} */ (row.attributes) + // Promoted keys and the event.name/event.timestamp identity leave the + // JSON; everything else stays. + assert.equal(attrs['session.id'], undefined) + assert.equal(attrs.tool_name, undefined) + assert.equal(attrs.decision, undefined) + assert.equal(attrs.source, undefined) + assert.equal(attrs['event.name'], undefined) + assert.equal(attrs['event.timestamp'], undefined) + assert.equal(attrs.language, 'javascript') + assert.equal(attrs['app.version'], '2.1.233') +}) + +test('cost_usd is typed from the string-typed numeric Claude Code sends', () => { + const rows = claudeTelemetryEventRows([ + event('api_request', { cost_usd: '0.0047732', input_tokens: 73, request_id: 'req_1' }), + ]) + assert.equal(rows.length, 1) + assert.equal(rows[0].cost_usd, 0.0047732) + const attrs = /** @type {Record} */ (rows[0].attributes) + assert.equal(attrs.cost_usd, undefined) + assert.equal(attrs.input_tokens, 73) + assert.equal(attrs.request_id, 'req_1') +}) + +test('an unrecognized event name is recorded with its attributes, not discarded', () => { + // @ref LLP 0257#failure-modes [tests]: unknown names keep their attributes + const rows = claudeTelemetryEventRows([ + event('brand_new_event', { detail: 'something upstream added' }), + ]) + assert.equal(rows.length, 1) + assert.equal(rows[0].event_name, 'brand_new_event') + const attrs = /** @type {Record} */ (rows[0].attributes) + assert.equal(attrs.detail, 'something upstream added') +}) + +test('a hot key whose value does not fit its typed column stays in the JSON instead of vanishing', () => { + const rows = claudeTelemetryEventRows([ + event('tool_decision', { tool_name: 'Read', decision: 42 }), + ]) + assert.equal(rows[0].decision, null) + const attrs = /** @type {Record} */ (rows[0].attributes) + assert.equal(attrs.decision, 42) +}) + +test('an event with no session id and no timestamp still lands, with nulls', () => { + const rows = claudeTelemetryEventRows([ + { name: 'mcp_server_connection', attributes: { server_name: 'some-mcp', status: 'connected' } }, + ]) + assert.equal(rows.length, 1) + assert.equal(rows[0].session_id, null) + assert.equal(rows[0].event_timestamp, null) + const attrs = /** @type {Record} */ (rows[0].attributes) + assert.equal(attrs.server_name, 'some-mcp') +}) + +// --------------------------------------------------------------------- +// flattenClaudeTelemetryMetrics: data points become events +// --------------------------------------------------------------------- + +/** @param {Record} attrs */ +function kv(attrs) { + return Object.entries(attrs).map(([key, value]) => { + if (typeof value === 'number') { + return Number.isInteger(value) + ? { key, value: { intValue: value } } + : { key, value: { doubleValue: value } } + } + if (typeof value === 'boolean') return { key, value: { boolValue: value } } + return { key, value: { stringValue: String(value) } } + }) +} + +const NANOS = String(BigInt(Date.parse('2026-08-17T19:31:00.000Z')) * 1_000_000n) + +/** + * @param {Record[]} metrics + * @param {Record} [resourceAttrs] + */ +function metricsEnvelope(metrics, resourceAttrs = { 'service.name': 'claude-code' }) { + return { + resourceMetrics: [ + { + resource: { attributes: kv(resourceAttrs) }, + scopeMetrics: [ + { scope: { name: 'com.anthropic.claude_code', version: '2.1.233' }, metrics }, + ], + }, + ], + } +} + +test('a sum data point becomes one event named by the metric, with value and unit joined', () => { + const events = flattenClaudeTelemetryMetrics(metricsEnvelope([ + { + name: 'claude_code.cost.usage', + unit: 'USD', + sum: { + aggregationTemporality: 2, + isMonotonic: true, + dataPoints: [ + { attributes: kv({ 'session.id': SESSION, model: 'claude-haiku-4-5-20251001' }), timeUnixNano: NANOS, asDouble: 0.0047732 }, + ], + }, + }, + ])) + assert.equal(events.length, 1) + assert.equal(events[0].name, 'claude_code.cost.usage') + assert.equal(events[0].timestamp, '2026-08-17T19:31:00.000Z') + assert.equal(events[0].attributes['session.id'], SESSION) + assert.equal(events[0].attributes.value, 0.0047732) + assert.equal(events[0].attributes.unit, 'USD') + assert.equal(events[0].attributes.model, 'claude-haiku-4-5-20251001') +}) + +test('an int64 value rendered as a string on the wire keeps numeric identity', () => { + const events = flattenClaudeTelemetryMetrics(metricsEnvelope([ + { + name: 'claude_code.lines_of_code.count', + sum: { aggregationTemporality: 2, isMonotonic: true, dataPoints: [ + { attributes: kv({ 'session.id': SESSION, type: 'added' }), timeUnixNano: NANOS, asInt: '42' }, + ] }, + }, + ])) + assert.equal(events.length, 1) + assert.equal(events[0].attributes.value, 42) + assert.equal(events[0].attributes.unit, undefined) +}) + +test('a gauge aggregation is read the same way a sum is', () => { + const events = flattenClaudeTelemetryMetrics(metricsEnvelope([ + { + name: 'claude_code.active_time.total', + unit: 's', + gauge: { dataPoints: [ + { attributes: kv({ 'session.id': SESSION }), timeUnixNano: NANOS, asDouble: 12.5 }, + ] }, + }, + ])) + assert.equal(events.length, 1) + assert.equal(events[0].attributes.value, 12.5) +}) + +test('metrics outside the claude scope and self-marked resources contribute nothing', () => { + const foreignScope = { + resourceMetrics: [ + { + resource: { attributes: kv({ 'service.name': 'someone-else' }) }, + scopeMetrics: [ + { scope: { name: 'io.other.meter' }, metrics: [ + { name: 'other.count', sum: { dataPoints: [{ attributes: [], timeUnixNano: NANOS, asInt: '1' }] } }, + ] }, + ], + }, + ], + } + assert.deepEqual(flattenClaudeTelemetryMetrics(foreignScope), []) + + const selfMarked = metricsEnvelope( + [{ name: 'claude_code.cost.usage', sum: { dataPoints: [{ attributes: [], timeUnixNano: NANOS, asDouble: 1 }] } }], + { 'service.name': 'hypaware', 'hypaware.self': true } + ) + assert.deepEqual(flattenClaudeTelemetryMetrics(selfMarked), []) +}) + +test('a malformed metrics envelope never throws', () => { + assert.deepEqual(flattenClaudeTelemetryMetrics(null), []) + assert.deepEqual(flattenClaudeTelemetryMetrics('nope'), []) + assert.deepEqual(flattenClaudeTelemetryMetrics({ resourceMetrics: [null, { scopeMetrics: [{ scope: {}, metrics: [{}] }] }] }), []) +}) + +// --------------------------------------------------------------------- +// The registration and its cache roundtrip +// --------------------------------------------------------------------- + +test('the registration names the dataset, its owner, its signal, and its timestamp column', () => { + const registration = claudeTelemetryDatasetRegistration() + assert.equal(registration.name, 'claude_telemetry_events') + assert.equal(registration.plugin, '@hypaware/claude') + assert.equal(registration.sourceSignal, 'claude_telemetry') + assert.equal(registration.primaryTimestampColumn, 'event_timestamp') + assert.equal(registration.localOnlyContentColumns, undefined) + assert.deepEqual( + registration.schema.columns.map((c) => c.name), + ['event_name', 'event_timestamp', 'session_id', 'tool_name', 'decision', 'source', 'cost_usd', 'attributes'] + ) +}) + +test('rows written through storage flush, discover, and read back through the registration', async () => { + const cacheRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-claude-events-')) + try { + const storage = createQueryStorageService({ cacheRoot }) + const rows = claudeTelemetryEventRows([ + event('tool_decision', { tool_name: 'Read', decision: 'accept', source: 'config' }), + event('permission_mode_changed', { from_mode: 'default', to_mode: 'acceptEdits' }), + ]) + const tablePath = claudeTelemetryTablePath(storage) + await storage.appendRows(tablePath, [...CLAUDE_TELEMETRY_EVENT_COLUMNS], rows) + await storage.flushTable(tablePath, { force: true }) + + const registration = claudeTelemetryDatasetRegistration() + const partitions = await registration.discoverPartitions( + /** @type {any} */ ({ config: { version: 2, plugins: [] }, scope: {}, cacheDir: cacheRoot }) + ) + assert.ok(partitions.length >= 2, 'spool partition plus the flushed source partition') + assert.equal(partitions[0].tablePath, path.join(cacheRoot, 'datasets', TELEMETRY_EVENTS_DATASET, 'all')) + + const source = await registration.createDataSource(partitions, /** @type {any} */ ({ scope: {}, storage })) + /** @type {Record[]} */ + const seen = [] + for await (const row of source.scan({}).rows()) { + if (/** @type {any} */ (row).resolved) seen.push(/** @type {any} */ (row).resolved) + } + assert.equal(seen.length, 2) + const names = seen.map((r) => r.event_name).sort() + assert.deepEqual(names, ['permission_mode_changed', 'tool_decision']) + const decision = seen.find((r) => r.event_name === 'tool_decision') + assert.ok(decision) + assert.equal(decision.decision, 'accept') + assert.equal(decision.source, 'config') + assert.equal(decision.session_id, SESSION) + } finally { + await fs.rm(cacheRoot, { recursive: true, force: true }) + } +}) diff --git a/test/plugins/claude-telemetry-listener.test.js b/test/plugins/claude-telemetry-listener.test.js new file mode 100644 index 00000000..576f315f --- /dev/null +++ b/test/plugins/claude-telemetry-listener.test.js @@ -0,0 +1,380 @@ +// @ts-check + +/** + * The Claude telemetry listener's deterministic halves: decoding the + * OTLP/JSON envelope Claude Code sends, and turning the decoded events + * into the same projected exchange the proxy and backfill producers + * yield. + * + * The event fixtures below are trimmed from a real capture (Claude Code + * 2.1.233, the LLP 0262 spike): same attribute names, same value + * wrappers, same string-typed numerics. + * + * @ref LLP 0257#testing [tests]: the deterministic parts (projection identity, + * config) are unit tested; the end-to-end seam is the hermetic smoke + */ + +import test from 'node:test' +import assert from 'node:assert/strict' + +import { + flattenClaudeTelemetryEvents, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/events.js' +import { + projectClaudeTelemetryEvents, + USAGE_INDEX_LIMIT, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/projection.js' +import { + DEFAULT_TELEMETRY_PORT, + partitionIgnoredSessionEvents, + readListenConfig, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/source.js' +import { validateClaudeConfig } from '../../hypaware-core/plugins-workspace/claude/src/config.js' +import { aiGatewayRowsFromProjectedExchange } from '../../hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js' + +const SESSION = 'e53c128d-9f45-470f-86f1-d5b5f3766708' +const PROMPT = '65cf592b-4153-482e-99a8-c22f1832b060' +const USER_UUID = '4bd39765-f83f-4a6f-bfc4-81b88f6ac446' +const ASSISTANT_UUID = '1e54d1be-9919-4b2a-97e2-3292ba55ce0e' +const REQUEST_ID = 'req_011Ce8sjpb8Uzvot2JMvFkKe' + +/** @param {Record} attrs */ +function kvAttributes(attrs) { + return Object.entries(attrs).map(([key, value]) => { + if (typeof value === 'number') { + return Number.isInteger(value) + ? { key, value: { intValue: value } } + : { key, value: { doubleValue: value } } + } + if (typeof value === 'boolean') return { key, value: { boolValue: value } } + return { key, value: { stringValue: String(value) } } + }) +} + +/** + * @param {string} name + * @param {Record} attrs + * @param {string} timestamp + */ +function record(name, attrs, timestamp) { + return { + timeUnixNano: String(BigInt(Date.parse(timestamp)) * 1_000_000n), + body: { stringValue: `claude_code.${name}` }, + attributes: kvAttributes({ + 'session.id': SESSION, + 'app.version': '2.1.233', + 'app.entrypoint': 'sdk-cli', + 'organization.id': '2efcd21e-aea6-42c6-9eda-a6e997ddcde4', + 'user.account_uuid': 'c9f39145-595f-4b31-9c66-c5c658a80aed', + 'terminal.type': 'ghostty', + 'event.name': name, + 'event.timestamp': timestamp, + 'prompt.id': PROMPT, + ...attrs, + }), + } +} + +/** @param {Array>} records */ +function envelope(records, resourceAttrs = { 'service.name': 'claude-code' }) { + return { + resourceLogs: [ + { + resource: { attributes: kvAttributes(resourceAttrs) }, + scopeLogs: [ + { + scope: { name: 'com.anthropic.claude_code.events', version: '2.1.233' }, + logRecords: records, + }, + ], + }, + ], + } +} + +function turnRecords() { + return [ + record('user_prompt', { + prompt_length: '86', + prompt: 'Run ls, then read notes.txt.', + 'message.uuid': USER_UUID, + }, '2026-08-17T19:30:24.450Z'), + record('api_request', { + model: 'claude-haiku-4-5-20251001', + input_tokens: 73, + output_tokens: 113, + cache_read_tokens: 35212, + cache_creation_tokens: 307, + cost_usd: 0.0047732, + duration_ms: 1842, + request_id: REQUEST_ID, + speed: 'normal', + query_source: 'sdk', + }, '2026-08-17T19:30:31.009Z'), + record('assistant_response', { + response_length: 93, + response: 'This is a spike repo.', + request_id: REQUEST_ID, + 'message.uuid': ASSISTANT_UUID, + model: 'claude-haiku-4-5-20251001', + query_source: 'sdk', + }, '2026-08-17T19:30:31.009Z'), + ] +} + +/** + * @param {Array>} records + * @param {Map>} [usage] + */ +function projectAll(records, usage = new Map()) { + return projectClaudeTelemetryEvents(flattenClaudeTelemetryEvents(envelope(records)), { + clientName: 'claude', + usageByRequestId: usage, + }) +} + +test('the OTLP envelope decodes to flat events keyed by event.name', () => { + const events = flattenClaudeTelemetryEvents(envelope(turnRecords())) + assert.deepEqual(events.map((e) => e.name), ['user_prompt', 'api_request', 'assistant_response']) + assert.equal(events[0].attributes['session.id'], SESSION) + assert.equal(events[0].timestamp, '2026-08-17T19:30:24.450Z') + // The AnyValue wrappers are gone: consumers see plain values. + assert.equal(events[1].attributes.input_tokens, 73) + assert.equal(events[1].attributes.cost_usd, 0.0047732) +}) + +test('a record with no event.name attribute contributes nothing', () => { + const bare = { timeUnixNano: '1786995009202000000', body: { stringValue: 'hello' }, attributes: [] } + assert.deepEqual(flattenClaudeTelemetryEvents(envelope([/** @type {any} */ (bare)])), []) +}) + +test('the timestamp falls back to timeUnixNano when event.timestamp is absent', () => { + const one = record('user_prompt', { prompt: 'hi', 'message.uuid': USER_UUID }, '2026-08-17T19:30:24.450Z') + one.attributes = one.attributes.filter((a) => a.key !== 'event.timestamp') + const [event] = flattenClaudeTelemetryEvents(envelope([one])) + assert.equal(event.timestamp, '2026-08-17T19:30:24.450Z') +}) + +test('a malformed envelope yields no events instead of throwing', () => { + assert.deepEqual(flattenClaudeTelemetryEvents(undefined), []) + assert.deepEqual(flattenClaudeTelemetryEvents({ resourceLogs: 'nope' }), []) + assert.deepEqual(flattenClaudeTelemetryEvents({ resourceLogs: [null, { scopeLogs: [{}] }] }), []) +}) + +test('the daemon\'s own telemetry is dropped, not ingested', () => { + const payload = envelope(turnRecords(), { 'service.name': 'hypaware-dev', 'hypaware.self': true }) + assert.deepEqual(flattenClaudeTelemetryEvents(payload), []) +}) + +test('another exporter\'s scope is ignored', () => { + const payload = envelope(turnRecords()) + payload.resourceLogs[0].scopeLogs[0].scope = { name: 'my.app', version: '1.0.0' } + assert.deepEqual(flattenClaudeTelemetryEvents(payload), []) +}) + +test('one turn projects to a user row and an assistant row with native uuids', () => { + const [projection] = projectAll(turnRecords()) + assert.equal(projection.provider, 'anthropic') + assert.equal(projection.session_id, SESSION) + assert.equal(projection.conversation_id, undefined) + assert.equal(projection.client_name, 'claude') + assert.equal(projection.conversation_source, 'claude_code') + assert.equal(projection.client_version, '2.1.233') + assert.equal(projection.entrypoint, 'sdk-cli') + assert.equal(projection.user_id, 'c9f39145-595f-4b31-9c66-c5c658a80aed') + + assert.equal(projection.messages.length, 2) + const [user, assistant] = projection.messages + assert.equal(user.role, 'user') + assert.equal(user.message_id, USER_UUID) + assert.equal(user.provider_uuid, USER_UUID) + assert.equal(user.content, 'Run ls, then read notes.txt.') + assert.equal(user.prompt_id, PROMPT) + + assert.equal(assistant.role, 'assistant') + assert.equal(assistant.message_id, ASSISTANT_UUID) + assert.equal(assistant.request_id, REQUEST_ID) + assert.equal(assistant.model, 'claude-haiku-4-5-20251001') +}) + +test('api_request usage lands on the assistant message it names', () => { + const [projection] = projectAll(turnRecords()) + const assistant = projection.messages[1] + assert.deepEqual(assistant.attributes?.usage, { + input_tokens: 73, + output_tokens: 113, + cache_read_tokens: 35212, + cache_write_tokens: 307, + }) + assert.equal(/** @type {any} */ (assistant.attributes)?.claude?.cost_usd, 0.0047732) +}) + +test('usage carries across batches, because the exporter flushes on a timer', () => { + const usage = new Map() + const [request, response] = [turnRecords()[1], turnRecords()[2]] + assert.deepEqual(projectAll([request], usage), []) + const [projection] = projectAll([response], usage) + assert.equal(/** @type {any} */ (projection.messages[0].attributes)?.usage?.output_tokens, 113) + // Claimed once: a later duplicate response does not re-read it. + assert.equal(usage.size, 0) +}) + +test('the usage index evicts oldest-first at its cap', () => { + const usage = new Map() + const records = [] + for (let i = 0; i < USAGE_INDEX_LIMIT + 5; i++) { + records.push(record('api_request', { request_id: `req-${i}`, output_tokens: i }, '2026-08-17T19:30:31.009Z')) + } + projectAll(records, usage) + assert.equal(usage.size, USAGE_INDEX_LIMIT) + assert.equal(usage.has('req-0'), false) + assert.equal(usage.has(`req-${USAGE_INDEX_LIMIT + 4}`), true) +}) + +test('a prompt event with content logging off produces no row', () => { + const noPrompt = record('user_prompt', { prompt_length: '86', 'message.uuid': USER_UUID }, '2026-08-17T19:30:24.450Z') + assert.deepEqual(projectAll([noPrompt]), []) +}) + +test('events for two sessions project to two exchanges', () => { + const other = record('user_prompt', { prompt: 'second', 'message.uuid': 'other-uuid' }, '2026-08-17T19:31:00.000Z') + other.attributes = other.attributes.map((a) => + a.key === 'session.id' ? { key: 'session.id', value: { stringValue: 'session-two' } } : a + ) + const projections = projectAll([...turnRecords(), other]) + assert.equal(projections.length, 2) + assert.deepEqual(projections.map((p) => p.session_id).sort(), [SESSION, 'session-two']) +}) + +test('cwd and git identity come from the SessionStart hook record', () => { + const events = flattenClaudeTelemetryEvents(envelope(turnRecords())) + const [projection] = projectClaudeTelemetryEvents(events, { + clientName: 'claude', + usageByRequestId: new Map(), + sessionContext: (id) => id === SESSION + ? { + session_id: id, + transcript_path: undefined, + cwd: '/repo', + git_branch: 'main', + git_remote: 'git@github.com:o/r.git', + head_sha: 'a'.repeat(40), + repo_root: '/repo', + ts: undefined, + } + : undefined, + }) + assert.equal(projection.cwd, '/repo') + assert.equal(projection.git_branch, 'main') + assert.equal(projection.repo_root, '/repo') +}) + +test('the expanded rows carry native part identity and null parent-chain columns', () => { + const [projection] = projectAll(turnRecords()) + const rows = aiGatewayRowsFromProjectedExchange(projection) + assert.equal(rows.length, 2) + assert.equal(rows[0].part_id, `${USER_UUID}#0`) + assert.equal(rows[1].part_id, `${ASSISTANT_UUID}#0`) + // Native identity: nothing here is a gateway fallback, so no settlement + // enricher has anything to repair. @ref LLP 0254#identity-at-ingest + for (const row of rows) { + assert.equal(/** @type {any} */ (row.attributes)?.gateway?.identity_source, undefined) + // @ref LLP 0252#consequences: these read null on the OTEL path by design. + assert.equal(row.parent_uuid, undefined) + assert.equal(row.logical_parent_uuid, undefined) + assert.equal(row.user_type, undefined) + assert.equal(row.permission_mode, undefined) + assert.equal(row.session_id, SESSION) + assert.equal(row.client_name, 'claude') + assert.equal(row.provider, 'anthropic') + } + assert.equal(rows[1].model, 'claude-haiku-4-5-20251001') + assert.deepEqual(rows[0].previous_message_id, []) + assert.deepEqual(rows[1].previous_message_id, [USER_UUID]) +}) + +test('replaying the same events re-expands to the same part ids', () => { + const first = aiGatewayRowsFromProjectedExchange(projectAll(turnRecords())[0]) + const second = aiGatewayRowsFromProjectedExchange(projectAll(turnRecords())[0]) + assert.deepEqual(first.map((r) => r.part_id), second.map((r) => r.part_id)) +}) + +// @ref LLP 0256#control-route-on-listener [tests]: ingest drops by session +// id against the listener's own in-memory set, on the same verbatim-token +// match the gateway applies (LLP 0066 R5). +test('an ignored session\'s events are partitioned out, keyed verbatim on session.id', () => { + const events = flattenClaudeTelemetryEvents(envelope(turnRecords())) + const { kept, droppedBySession } = partitionIgnoredSessionEvents(events, new Set([SESSION])) + assert.deepEqual(kept, []) + assert.equal(droppedBySession.size, 1) + assert.equal(droppedBySession.get(SESSION)?.length, 3) + + // The token is opaque and never normalized: a trimmed or case-shifted + // variant of the id matches nothing, so those events are recorded. + const nearMiss = partitionIgnoredSessionEvents(events, new Set([` ${SESSION} `, SESSION.toUpperCase()])) + assert.equal(nearMiss.kept.length, 3) + assert.equal(nearMiss.droppedBySession.size, 0) +}) + +test('only the ignored session drops; other sessions and unattributed events are kept', () => { + const other = record('user_prompt', { prompt: 'second', 'message.uuid': 'other-uuid' }, '2026-08-17T19:31:00.000Z') + other.attributes = other.attributes.map((a) => + a.key === 'session.id' ? { key: 'session.id', value: { stringValue: 'session-two' } } : a + ) + // An event naming NO session cannot match an exact key, so it is kept: + // dropping it would suppress rows nobody opted out. + const anonymous = record('user_prompt', { prompt: 'third', 'message.uuid': 'anon-uuid' }, '2026-08-17T19:32:00.000Z') + anonymous.attributes = anonymous.attributes.filter((a) => a.key !== 'session.id') + + const events = flattenClaudeTelemetryEvents(envelope([...turnRecords(), other, anonymous])) + const { kept, droppedBySession } = partitionIgnoredSessionEvents(events, new Set(['session-two'])) + assert.equal(kept.length, 4) + assert.deepEqual([...droppedBySession.keys()], ['session-two']) + assert.equal(droppedBySession.get('session-two')?.length, 1) +}) + +test('an empty ignore set keeps every event and allocates no buckets', () => { + const events = flattenClaudeTelemetryEvents(envelope(turnRecords())) + const { kept, droppedBySession } = partitionIgnoredSessionEvents(events, new Set()) + assert.equal(kept.length, events.length) + assert.equal(droppedBySession.size, 0) +}) + +test('the listener config defaults to loopback on its own port', () => { + const warnings = [] + const ctx = /** @type {any} */ ({ config: {}, log: { warn: (/** @type {any} */ m) => warnings.push(m) } }) + assert.deepEqual(readListenConfig(ctx), { + host: '127.0.0.1', + port: DEFAULT_TELEMETRY_PORT, + portConfigured: false, + }) + assert.equal(warnings.length, 0) +}) + +test('a configured port is marked configured, so it never silently falls back', () => { + const ctx = /** @type {any} */ ({ + config: { telemetry: { listen_host: '127.0.0.2', listen_port: 0 } }, + log: { warn: () => {} }, + }) + assert.deepEqual(readListenConfig(ctx), { host: '127.0.0.2', port: 0, portConfigured: true }) +}) + +test('a mistyped listener port warns and falls back to the default', () => { + const warnings = [] + const ctx = /** @type {any} */ ({ + config: { telemetry: { listen_port: '4319' } }, + log: { warn: (/** @type {any} */ m) => warnings.push(m) }, + }) + assert.equal(readListenConfig(ctx).port, DEFAULT_TELEMETRY_PORT) + assert.equal(warnings.length, 1) +}) + +test('the telemetry config block is validated', () => { + assert.equal(validateClaudeConfig({ telemetry: { listen_host: '127.0.0.1', listen_port: 4319 } }).ok, true) + const badPort = validateClaudeConfig({ telemetry: { listen_port: 70000 } }) + assert.equal(badPort.ok, false) + assert.equal(badPort.errors?.[0].pointer, '/telemetry/listen_port') + const typo = validateClaudeConfig({ telemetry: { listen_ports: 4319 } }) + assert.equal(typo.ok, false) + assert.equal(typo.errors?.[0].pointer, '/telemetry/listen_ports') +}) diff --git a/test/plugins/claude-telemetry-spool.test.js b/test/plugins/claude-telemetry-spool.test.js new file mode 100644 index 00000000..0ca7690c --- /dev/null +++ b/test/plugins/claude-telemetry-spool.test.js @@ -0,0 +1,196 @@ +// @ts-check + +/** + * The body spool's deterministic contract: the fixed path under the + * HypAware home, owner-only creation, and the byte cap with strictly + * oldest-first eviction. + * + * @ref LLP 0257#testing [tests]: spool cap eviction order is unit tested in the + * root suite + */ + +import assert from 'node:assert/strict' +import fsp from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import test from 'node:test' + +import { + DEFAULT_SPOOL_MAX_BYTES, + claudeBodySpoolDir, + enforceClaudeBodySpoolCap, + ensureClaudeBodySpool, + tightenClaudeBodySpool, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/spool.js' +import { + readSpoolConfig, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/source.js' +import { validateClaudeConfig } from '../../hypaware-core/plugins-workspace/claude/src/config.js' + +async function tmpSpool() { + const root = await fsp.mkdtemp(path.join(os.tmpdir(), 'hyp-spool-')) + return path.join(root, 'spool', 'claude-bodies') +} + +/** + * @param {string} dir + * @param {string} name + * @param {number} size + * @param {string} mtime ISO timestamp + */ +async function spoolFile(dir, name, size, mtime) { + const file = path.join(dir, name) + await fsp.writeFile(file, Buffer.alloc(size, 0x61)) + const when = new Date(mtime) + await fsp.utimes(file, when, when) + return file +} + +/** @param {string} dir */ +async function names(dir) { + return (await fsp.readdir(dir)).sort() +} + +test('the spool path is fixed under the HypAware home', () => { + assert.equal(claudeBodySpoolDir('/home/u/.hyp'), path.join('/home/u/.hyp', 'spool', 'claude-bodies')) +}) + +test('ensureClaudeBodySpool creates the directory owner-only', async () => { + const dir = await tmpSpool() + await ensureClaudeBodySpool(dir) + const stat = await fsp.stat(dir) + assert.equal(stat.mode & 0o777, 0o700) +}) + +/** + * The daemon's half of the same duty. Attach is what mints the spool (it is + * the same write that tells Claude Code where to put bodies), so the listener + * repairs a directory it finds and creates none: a daemon that minted one + * anyway would leave a raw-prompt directory on every install that never + * attached this client, under whatever HYP_HOME the activation context + * resolved - which is how a test run reaches a developer's real `~/.hyp`. + * + * @ref LLP 0253#spool-location [tests]: the daemon keeps the directory owner-only without minting it + */ +test('tightenClaudeBodySpool repairs an existing spool and creates nothing', async () => { + const dir = await tmpSpool() + + assert.equal(await tightenClaudeBodySpool(dir), false) + await assert.rejects(fsp.stat(dir), /ENOENT/) + + await fsp.mkdir(dir, { recursive: true, mode: 0o755 }) + await fsp.chmod(dir, 0o755) + assert.equal(await tightenClaudeBodySpool(dir), true) + assert.equal((await fsp.stat(dir)).mode & 0o777, 0o700) +}) + +test('an existing spool with loose permissions is tightened, not trusted', async () => { + const dir = await tmpSpool() + // Claude Code creates the directory itself when it writes the first + // body before the daemon ever ran; that copy gets the default umask. + await fsp.mkdir(dir, { recursive: true, mode: 0o755 }) + await fsp.chmod(dir, 0o755) + await ensureClaudeBodySpool(dir) + const stat = await fsp.stat(dir) + assert.equal(stat.mode & 0o777, 0o700) +}) + +test('a spool under its cap is left alone', async () => { + const dir = await tmpSpool() + await ensureClaudeBodySpool(dir) + await spoolFile(dir, 'a.request.json', 100, '2026-08-17T10:00:00Z') + await spoolFile(dir, 'b.request.json', 100, '2026-08-17T11:00:00Z') + const result = await enforceClaudeBodySpoolCap(dir, 500) + assert.deepEqual(result, { spoolBytes: 200, evictedCount: 0, evictedBytes: 0 }) + assert.deepEqual(await names(dir), ['a.request.json', 'b.request.json']) +}) + +test('eviction removes strictly the oldest files until the total fits', async () => { + const dir = await tmpSpool() + await ensureClaudeBodySpool(dir) + // Write in an order unrelated to age so mtime, not creation order or + // directory order, decides who goes. + await spoolFile(dir, 'newest.response.json', 100, '2026-08-17T12:00:00Z') + await spoolFile(dir, 'oldest.request.json', 300, '2026-08-17T09:00:00Z') + await spoolFile(dir, 'middle.request.json', 200, '2026-08-17T10:30:00Z') + + const result = await enforceClaudeBodySpoolCap(dir, 350) + // 600 bytes over a 350 cap: oldest (300) goes first, leaving 300, + // which fits; middle and newest survive. + assert.deepEqual(result, { spoolBytes: 300, evictedCount: 1, evictedBytes: 300 }) + assert.deepEqual(await names(dir), ['middle.request.json', 'newest.response.json']) + + const again = await enforceClaudeBodySpoolCap(dir, 120) + // Still oldest-first: middle (200) goes before newest (100). + assert.deepEqual(again, { spoolBytes: 100, evictedCount: 1, evictedBytes: 200 }) + assert.deepEqual(await names(dir), ['newest.response.json']) +}) + +test('a cap smaller than every file empties the spool oldest-first', async () => { + const dir = await tmpSpool() + await ensureClaudeBodySpool(dir) + await spoolFile(dir, 'one.request.json', 100, '2026-08-17T09:00:00Z') + await spoolFile(dir, 'two.request.json', 100, '2026-08-17T10:00:00Z') + const result = await enforceClaudeBodySpoolCap(dir, 50) + assert.deepEqual(result, { spoolBytes: 0, evictedCount: 2, evictedBytes: 200 }) + assert.deepEqual(await names(dir), []) +}) + +test('tied mtimes fall back to name order, so eviction stays deterministic', async () => { + const dir = await tmpSpool() + await ensureClaudeBodySpool(dir) + await spoolFile(dir, 'b-second.json', 100, '2026-08-17T09:00:00Z') + await spoolFile(dir, 'a-first.json', 100, '2026-08-17T09:00:00Z') + const result = await enforceClaudeBodySpoolCap(dir, 150) + assert.equal(result.evictedCount, 1) + assert.deepEqual(await names(dir), ['b-second.json']) +}) + +test('a missing spool directory reads as empty, not as an error', async () => { + const dir = path.join(await tmpSpool(), 'never-created') + const result = await enforceClaudeBodySpoolCap(dir, 100) + assert.deepEqual(result, { spoolBytes: 0, evictedCount: 0, evictedBytes: 0 }) +}) + +test('the spool config resolves the fixed dir and defaults the cap to 512 MB', () => { + const ctx = /** @type {any} */ ({ + env: { HYP_HOME: '/tmp/hyp-home' }, + config: {}, + log: { warn: () => {} }, + }) + const spool = readSpoolConfig(ctx) + assert.equal(spool.dir, path.join('/tmp/hyp-home', 'spool', 'claude-bodies')) + assert.equal(spool.maxBytes, DEFAULT_SPOOL_MAX_BYTES) + assert.equal(DEFAULT_SPOOL_MAX_BYTES, 512 * 1024 * 1024) +}) + +test('a configured cap wins, and a mistyped one warns and falls back', () => { + const warnings = /** @type {any[]} */ ([]) + const good = /** @type {any} */ ({ + env: { HYP_HOME: '/tmp/hyp-home' }, + config: { telemetry: { spool_max_bytes: 65536 } }, + log: { warn: (/** @type {any} */ m) => warnings.push(m) }, + }) + assert.equal(readSpoolConfig(good).maxBytes, 65536) + assert.equal(warnings.length, 0) + + const bad = /** @type {any} */ ({ + env: { HYP_HOME: '/tmp/hyp-home' }, + config: { telemetry: { spool_max_bytes: '512mb' } }, + log: { warn: (/** @type {any} */ m) => warnings.push(m) }, + }) + assert.equal(readSpoolConfig(bad).maxBytes, DEFAULT_SPOOL_MAX_BYTES) + assert.equal(warnings.length, 1) +}) + +test('the config validator accepts a positive integer cap and rejects the rest', () => { + assert.equal(validateClaudeConfig({ telemetry: { spool_max_bytes: 1024 } }).ok, true) + const zero = validateClaudeConfig({ telemetry: { spool_max_bytes: 0 } }) + assert.equal(zero.ok, false) + assert.equal(zero.errors?.[0].pointer, '/telemetry/spool_max_bytes') + const fractional = validateClaudeConfig({ telemetry: { spool_max_bytes: 1.5 } }) + assert.equal(fractional.ok, false) + const typo = validateClaudeConfig({ telemetry: { spool_max_byte: 1024 } }) + assert.equal(typo.ok, false) + assert.equal(typo.errors?.[0].pointer, '/telemetry/spool_max_byte') +}) diff --git a/test/plugins/claude-telemetry-start-failure.test.js b/test/plugins/claude-telemetry-start-failure.test.js new file mode 100644 index 00000000..d7c1ddbe --- /dev/null +++ b/test/plugins/claude-telemetry-start-failure.test.js @@ -0,0 +1,98 @@ +// @ts-check + +/** + * A listener start that cannot bind must leave nothing running. + * + * The spool sweep is on a repeating timer, and `stop()` is the only thing that + * clears it - but a `start()` that throws never returns a handle to call + * `stop()` on. Arming the timer before the bind therefore left it scanning the + * spool once a minute, for the life of the daemon, on behalf of a source that + * does not exist. `unref()` keeps that from holding the process open, which is + * exactly why it would never have been noticed. + * + * @ref LLP 0114#explicit-listen-fails-loudly [tests]: a configured port that is + * taken is a loud source-start failure, and a failed start owns nothing + */ + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import net from 'node:net' +import os from 'node:os' +import path from 'node:path' + +import { createStartClaudeTelemetrySource } from '../../hypaware-core/plugins-workspace/claude/src/telemetry/source.js' + +/** + * Count `setInterval` calls for the duration of `fn`. An `unref`'d timer does + * not show up in `process.getActiveResourcesInfo()`, which is precisely why an + * orphaned one would never be noticed at runtime, so the arming itself is what + * the test observes. + * + * @param {() => Promise} fn + * @returns {Promise<{ armed: number, error: unknown }>} + */ +async function countIntervals(fn) { + const original = globalThis.setInterval + let armed = 0 + /** @type {unknown} */ + let error + globalThis.setInterval = /** @type {typeof globalThis.setInterval} */ ( + /** @type {unknown} */ ((/** @type {any[]} */ ...args) => { + armed += 1 + return /** @type {any} */ (original)(...args) + }) + ) + try { + await fn() + } catch (err) { + error = err + } finally { + globalThis.setInterval = original + } + return { armed, error } +} + +/** A listener holding a port so the configured bind below cannot have it. */ +function occupy() { + return new Promise((resolve) => { + const server = net.createServer() + server.listen(0, '127.0.0.1', () => { + const address = server.address() + const port = typeof address === 'object' && address ? address.port : 0 + resolve({ port, close: () => new Promise((r) => server.close(() => r(undefined))) }) + }) + }) +} + +test('a listener that cannot bind its configured port leaves no sweep timer behind', async () => { + const hypHome = await fs.mkdtemp(path.join(os.tmpdir(), 'hyp-claude-start-fail-')) + const taken = /** @type {{ port: number, close: () => Promise }} */ (await occupy()) + try { + const start = createStartClaudeTelemetrySource({ + gateway: /** @type {any} */ ({ recordProjectedExchange: async () => ({ written: 0 }) }), + clientName: 'claude', + stateFile: path.join(hypHome, 'claude-sessions.json'), + }) + const noop = () => {} + const ctx = /** @type {any} */ ({ + // An EXPLICIT port, so `bindWithFallback` refuses to fall back and the + // start really does throw (LLP 0114 §explicit-listen-fails-loudly). + config: { telemetry: { listen_host: '127.0.0.1', listen_port: taken.port } }, + env: { HYP_HOME: hypHome }, + log: { info: noop, warn: noop, error: noop, debug: noop }, + storage: {}, + }) + + const { armed, error } = await countIntervals(() => start(ctx)) + assert.ok(error, 'the configured port was taken, so the start must fail') + assert.equal( + armed, + 0, + 'the failed start armed a repeating timer with no stop() left to clear it' + ) + } finally { + await taken.close() + await fs.rm(hypHome, { recursive: true, force: true }) + } +}) diff --git a/test/plugins/claude-telemetry-usage-policy.test.js b/test/plugins/claude-telemetry-usage-policy.test.js new file mode 100644 index 00000000..d94bbe62 --- /dev/null +++ b/test/plugins/claude-telemetry-usage-policy.test.js @@ -0,0 +1,214 @@ +// @ts-check + +/** + * The folder usage policy on the OTEL ingest path: the check that runs before + * a row exists, rather than the flush-time late drop the proxy path needs. + * + * Three outcomes are pinned here: an `ignore` cwd is dropped, a `local-only` + * or `full` cwd is recorded (its withholding happens at the export and query + * seams), and a session whose cwd nothing recorded is withheld rather than + * treated as clean. + * + * @ref LLP 0254#policy-inline [tests]: the verdict is in hand before the write, + * so the LLP 0085 fail-open window has nothing to reopen + * @ref LLP 0257#ingest [tests]: S10 - a session with no hook record is + * undetermined, not clean + */ + +import test from 'node:test' +import assert from 'node:assert/strict' +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' + +import { + createUsagePolicyResolver, + localOnlyListPath, + writeLocalOnlyEntries, +} from '../../src/core/usage-policy/index.js' +import { + POLICY_UNDETERMINED, + partitionByUsagePolicy, + resolveSessionUsagePolicy, +} from '../../hypaware-core/plugins-workspace/claude/src/telemetry/policy.js' + +/** + * @import { ClaudeTelemetryEvent, SessionContextRecord } from '../../hypaware-core/plugins-workspace/claude/src/types.js' + */ + +/** + * @param {string} name + * @param {string | undefined} sessionId + * @param {Record} [attrs] + * @returns {ClaudeTelemetryEvent} + */ +function event(name, sessionId, attrs = {}) { + return { + name, + timestamp: '2026-08-17T20:30:24.450Z', + attributes: { + ...(sessionId === undefined ? {} : { 'session.id': sessionId }), + ...attrs, + }, + } +} + +/** + * @param {string} sessionId + * @param {string | undefined} cwd + * @returns {SessionContextRecord} + */ +function hookRecord(sessionId, cwd) { + return { + session_id: sessionId, + transcript_path: undefined, + cwd, + git_branch: undefined, + ts: '2026-08-17T20:30:00.000Z', + } +} + +/** @param {string} prefix */ +function tmpDir(prefix) { + return fs.mkdtemp(path.join(os.tmpdir(), `hyp-otel-policy-${prefix}-`)) +} + +/* ------------------------- resolveSessionUsagePolicy ------------------------ */ + +test('resolveSessionUsagePolicy: an ancestor .hypignore resolves the session to ignore', async () => { + const root = await tmpDir('hypignore') + try { + const repo = path.join(root, 'secret-repo') + await fs.mkdir(path.join(repo, 'sub'), { recursive: true }) + const governing = path.join(repo, '.hypignore') + await fs.writeFile(governing, 'ignore\n') + + const verdict = resolveSessionUsagePolicy({ + record: hookRecord('s1', path.join(repo, 'sub')), + resolver: createUsagePolicyResolver(), + }) + assert.equal(verdict.class, 'ignore') + assert.equal(verdict.governedBy, governing) + assert.equal(verdict.cwd, path.join(repo, 'sub')) + } finally { + await fs.rm(root, { recursive: true, force: true }) + } +}) + +test('resolveSessionUsagePolicy: the machine-local list governs a directory with no dotfile', async () => { + const root = await tmpDir('machine-local') + try { + const stateDir = path.join(root, 'hypaware') + const repo = path.join(root, 'private-repo') + await fs.mkdir(repo, { recursive: true }) + // The list the picker and `hyp ignore --private` write: no `.hypignore` + // exists anywhere near this directory, so a dotfile-only view would call + // it `full`. + await writeLocalOnlyEntries({ stateDir, entries: [{ dir: repo, class: 'ignore' }] }) + + const resolver = createUsagePolicyResolver({ localOnlyListPath: localOnlyListPath(stateDir) }) + const verdict = resolveSessionUsagePolicy({ record: hookRecord('s1', repo), resolver }) + assert.equal(verdict.class, 'ignore') + assert.equal(verdict.governedBy, localOnlyListPath(stateDir)) + + // The same session, resolved by a resolver that was never told where the + // list lives (the per-plugin state dir has no list file), reads clean. + // That is the failure this wiring exists to prevent. + const blind = createUsagePolicyResolver({ + localOnlyListPath: localOnlyListPath(path.join(stateDir, 'plugins', '@hypaware/claude')), + }) + assert.equal(resolveSessionUsagePolicy({ record: hookRecord('s1', repo), resolver: blind }).class, 'full') + } finally { + await fs.rm(root, { recursive: true, force: true }) + } +}) + +test('resolveSessionUsagePolicy: no hook record, or a record with no cwd, is undetermined', () => { + const resolver = createUsagePolicyResolver() + assert.equal(resolveSessionUsagePolicy({ record: undefined, resolver }).class, POLICY_UNDETERMINED) + assert.equal( + resolveSessionUsagePolicy({ record: hookRecord('s1', undefined), resolver }).class, + POLICY_UNDETERMINED + ) +}) + +test('resolveSessionUsagePolicy: local-only is not a drop', async () => { + const root = await tmpDir('local-only') + try { + const repo = path.join(root, 'repo') + await fs.mkdir(repo, { recursive: true }) + await fs.writeFile(path.join(repo, '.hypignore'), 'local-only\n') + const verdict = resolveSessionUsagePolicy({ + record: hookRecord('s1', repo), + resolver: createUsagePolicyResolver(), + }) + assert.equal(verdict.class, 'local-only') + } finally { + await fs.rm(root, { recursive: true, force: true }) + } +}) + +/* --------------------------- partitionByUsagePolicy -------------------------- */ + +/** @param {Record} bySession */ +function verdictTable(bySession) { + /** @param {string} sessionId */ + return (sessionId) => { + const cls = bySession[sessionId] ?? POLICY_UNDETERMINED + return cls === POLICY_UNDETERMINED + ? /** @type {const} */ ({ class: POLICY_UNDETERMINED }) + : { class: cls, cwd: `/w/${sessionId}`, governedBy: `/w/${sessionId}/.hypignore`, declared: cls } + } +} + +test('partitionByUsagePolicy: ignored sessions are dropped, everything else is kept', () => { + const events = [ + event('user_prompt', 'ignored', { prompt: 'secret' }), + event('api_request_body', 'ignored', { body_ref: '/spool/ignored.json' }), + event('user_prompt', 'clean'), + event('user_prompt', 'local'), + ] + const split = partitionByUsagePolicy(events, { + verdictFor: verdictTable({ ignored: 'ignore', clean: 'full', local: 'local-only' }), + }) + + assert.deepEqual(split.kept.map((e) => e.attributes['session.id']), ['clean', 'local']) + assert.deepEqual([...split.droppedBySession.keys()], ['ignored']) + assert.equal(split.droppedBySession.get('ignored')?.events.length, 2) + assert.equal(split.droppedBySession.get('ignored')?.verdict.class, 'ignore') + assert.equal(split.withheldBySession.size, 0) +}) + +test('partitionByUsagePolicy: an undetermined session is withheld, not kept', () => { + const split = partitionByUsagePolicy( + [event('user_prompt', 'unknown'), event('api_request', 'unknown'), event('user_prompt', 'clean')], + { verdictFor: verdictTable({ clean: 'full' }) } + ) + + assert.deepEqual(split.kept.map((e) => e.attributes['session.id']), ['clean']) + assert.equal(split.droppedBySession.size, 0) + assert.equal(split.withheldBySession.get('unknown')?.events.length, 2) +}) + +test('partitionByUsagePolicy: an event naming no session is kept', () => { + const split = partitionByUsagePolicy([event('tool_decision', undefined)], { + verdictFor: verdictTable({}), + }) + assert.equal(split.kept.length, 1) + assert.equal(split.withheldBySession.size, 0) +}) + +test('partitionByUsagePolicy: each session is resolved once per batch', () => { + let calls = 0 + const split = partitionByUsagePolicy( + [event('user_prompt', 's'), event('api_request', 's'), event('assistant_response', 's')], + { + verdictFor: (sessionId) => { + calls += 1 + return verdictTable({ s: 'full' })(sessionId) + }, + } + ) + assert.equal(calls, 1) + assert.equal(split.kept.length, 3) +}) diff --git a/test/plugins/claude-version.test.js b/test/plugins/claude-version.test.js new file mode 100644 index 00000000..190c4067 --- /dev/null +++ b/test/plugins/claude-version.test.js @@ -0,0 +1,77 @@ +// @ts-check + +import assert from 'node:assert/strict' +import test from 'node:test' + +import { + CLAUDE_OTEL_MIN_VERSION, + compareClaudeVersions, + detectClaudeCodeVersion, + isBelowClaudeVersion, + parseClaudeVersion, + resolveClaudeCodeVersion, +} from '../../hypaware-core/plugins-workspace/claude/src/claude_version.js' + +test('parseClaudeVersion pulls the numeric triple out of the CLI banner', () => { + assert.equal(parseClaudeVersion('2.1.233 (Claude Code)'), '2.1.233') + assert.equal(parseClaudeVersion('Claude Code v2.1.193\n'), '2.1.193') + assert.equal(parseClaudeVersion('no version here'), undefined) + assert.equal(parseClaudeVersion(undefined), undefined) + assert.equal(parseClaudeVersion(42), undefined) +}) + +// The trap the numeric compare exists for: lexically '2.1.193' < '2.1.9', +// which would refuse exactly the releases that clear the floor. +test('compareClaudeVersions is numeric, not lexical', () => { + assert.ok(compareClaudeVersions('2.1.193', '2.1.9') > 0) + assert.ok(compareClaudeVersions('2.1.9', '2.1.193') < 0) + assert.equal(compareClaudeVersions('2.1.193', '2.1.193'), 0) + assert.ok(compareClaudeVersions('3.0.0', '2.99.99') > 0) +}) + +// @ref LLP 0258#version-floor [tests]: *older than* the floor refuses; nothing else does +test('isBelowClaudeVersion: only a version proven older than the floor is below', () => { + assert.equal(isBelowClaudeVersion('2.1.192'), true) + assert.equal(isBelowClaudeVersion(CLAUDE_OTEL_MIN_VERSION), false) + assert.equal(isBelowClaudeVersion('2.1.233'), false) + // Unknown is not old. + assert.equal(isBelowClaudeVersion(undefined), false) + assert.equal(isBelowClaudeVersion('not-a-version'), false) +}) + +test('detectClaudeCodeVersion parses the probe output and never throws', async () => { + const detected = await detectClaudeCodeVersion({ + exec: /** @type {any} */ (async () => ({ stdout: '2.1.233 (Claude Code)' })), + }) + assert.equal(detected, '2.1.233') + + const missing = await detectClaudeCodeVersion({ + exec: /** @type {any} */ (async () => { + throw new Error('ENOENT') + }), + }) + assert.equal(missing, undefined) +}) + +test('resolveClaudeCodeVersion: the env override wins without probing', async () => { + let probed = false + const version = await resolveClaudeCodeVersion( + { HYP_CLAUDE_CODE_VERSION: '2.1.200' }, + { + exec: /** @type {any} */ (async () => { + probed = true + return { stdout: '9.9.9' } + }), + } + ) + assert.equal(version, '2.1.200') + assert.equal(probed, false) +}) + +test('resolveClaudeCodeVersion: an unparseable override falls back to the probe', async () => { + const version = await resolveClaudeCodeVersion( + { HYP_CLAUDE_CODE_VERSION: 'whatever' }, + { exec: /** @type {any} */ (async () => ({ stdout: '2.1.233' })) } + ) + assert.equal(version, '2.1.233') +}) diff --git a/test/plugins/client-skill-manifest-agreement.test.js b/test/plugins/client-skill-manifest-agreement.test.js index edb0ae51..9ae497fe 100644 --- a/test/plugins/client-skill-manifest-agreement.test.js +++ b/test/plugins/client-skill-manifest-agreement.test.js @@ -55,6 +55,8 @@ async function registeredSkills(client) { agents: { register: noop }, commands: { register: noop }, backfills: { register: noop }, + sources: { register: noop }, + query: { registerDataset: noop }, configRegistry: { registerSection: noop }, initPresets: { register: noop }, paths: { stateDir: '/tmp/hyp-test-state' }, diff --git a/test/plugins/codex-privacy-skill-session-id.test.js b/test/plugins/codex-privacy-skill-session-id.test.js index 4b3087aa..b6f4976c 100644 --- a/test/plugins/codex-privacy-skill-session-id.test.js +++ b/test/plugins/codex-privacy-skill-session-id.test.js @@ -14,7 +14,7 @@ import { fileURLToPath } from 'node:url' * `opt-out confirmed`, and then the review discusses the machine's most * sensitive content believing it is not being recorded. The control route holds * the id as an opaque token and answers `ignored: true` for whatever it was - * handed (`ai-gateway/src/control.js`), so nothing downstream can catch a + * handed (`src/core/control/session_ignore.js`), so nothing downstream can catch a * wrong id. The correctness of the whole step rests on which id the embedded * script sends, which is a property of a markdown code block that no other * test covers.