Repository navigation
feat(supervisor): add WebSocket message introspection middleware #2428
Description
Activity
- added a parent issue
on Jul 22, 2026 🏗️ build-plan
Implementation Plan
Issue type:
feat
Complexity: High
Confidence: Medium — all design decisions resolved; residual risk is concentrated in the RFC 6455 parser refactorSummary
Add policy-selected supervisor middleware hooks for complete post-upgrade WebSocket messages in both directions, via a new bidirectional streaming gRPC operation. OpenShell keeps full ownership of RFC 6455 mechanics; middleware sees only complete logical messages. Delivered as two pull requests split at the return path, so the contract lands and gets exercised before the parser refactor begins.
Phase naming: this plan uses
WEBSOCKET_MESSAGE/PRE_CREDENTIALSandWEBSOCKET_MESSAGE/PRE_RETURNrather than the RFC 0009 working namesbefore_forward/before_return, to match the existing phase vocabulary and make the credential boundary explicit in the name. RFC 0009 gets updated to match.Scope
PR 1 — contract and forward path (runs on the existing, already-reviewed client parser; no RFC 6455 work)
proto/supervisor_middleware.proto: bidirectional streamingEvaluateWebSocket, typedpreflight/session_start/message/session_endrequest variants and their result variantscrates/openshell-supervisor-middleware/: binding-aware chain description, direction-scoped session runner, concurrency admission, deadline propagationcrates/openshell-supervisor-network/src/l7/: forward-direction text integration and preflight resolution at the upgrade-admission pointcrates/openshell-server/src/grpc/,crates/openshell-supervisor-network/src/opa.rs: generalize manifest validation from the single hard-codedHTTP_REQUEST/PRE_CREDENTIALSpair to an allowlist
PR 2 — return path and binary
- Direction-aware frame validation, binary reassembly, upgrade-response overflow routing
permessage-deflatestripping for inspected sessions, and the new extension mode it requires- Full OCSF coverage and user-facing documentation
No changes to the policy YAML schema,
MiddlewareServiceFileConfig, orproto/sandbox.proto. Every limit introduced here is a platform constant; every option that would have required an operator field was deliberately deferred.Implementation Steps
- High-level failing test through the real relay, written in its final form including return-path assertions, ignored until PR 2. Fixture is controllably hostile from the start (slow, hanging, mid-message close, out-of-order, oversize) driven by simulated time
- Generalize endpoint attachment across bindings, preserving host selection, global ordering, and all existing HTTP behavior
- Extend the contract and registration; add
max_message_bytes, deadline propagation, and the streaming RPC - Build the direction-scoped chain runner: preflight, sequencing, limits, failure modes, concurrency admission
- (PR 2) Refactor RFC 6455 parsing around logical messages, both directions
- OCSF coverage
- Documentation, RFC 0009 update, and skill review
Test Plan
- Unit: manifest and binding validation, envelope limits, sequencing, decision semantics, transformation validity, per-stage failure modes under both
on_errorsettings, permit contention, chain budget expiry - Integration: full relay upgrade with an in-process streaming middleware — transformation, denial, all-skip preflight, mixed preflight, fragmented messages, control-frame interleaving, close-code correctness, credential-boundary ordering, and OCSF non-leakage using sentinel payloads
- E2E:
mise run e2e:docker, since this changes sandbox network infrastructure - Known gap: no fuzz or conformance coverage of the new parser at ship time. Filed as a follow-up rather than gating PR 2, since it likely needs the parser factored to run off a byte slice
Key Design Decisions
Resolved during review. Each of these was a genuine fork:
- Compression: strip
permessage-deflatewhen any stage inspects, rather than teaching the proxy to decompress and recompress. An upstream that selects a non-offered extension hard-fails the upgrade — every tolerant alternative hands the upstream a unilateral inspection bypass - Blocking model: middleware evaluation blocks its direction including control frames, keeping the existing sequential loop. Bounded by the per-message deadline plus a new total chain budget. Accepted rough edge: a long deadline can break peer keepalive, documented rather than engineered around
- Close codes: middleware deny and middleware failure share
1008. Distinguishing them lets a workload detect when inspection is degraded, which underfail_openis exactly when its traffic goes unexamined.1002/1009/1012stay distinct because conforming clients need them - Disclosure posture: failure detail is opaque to the sandbox and precise in OCSF.
reason_codeis recorded, never surfaced to the workload - Oversize: closes
1009regardless ofon_error. Routing it through fail-open would let a workload escape inspection by exceeding a known threshold - Resource bound: a semaphore on the shared middleware registry admits one unit of work per message or per preflight fan-out. Permit wait is backpressure excluded from the deadline, so load can never manufacture a timeout. This is a concurrency and memory bound, explicitly not rate limiting
Risks & Open Questions
- Parser surface. Upstream bytes go from an opaque copy to a parsed, attacker-reachable input. Mitigated by keeping parsing independent of middleware, preserving the raw fast path, and the fuzz follow-up
- Behavior change. Binary messages between 4 MiB and 16 MiB relay fine today and will fail with
1009once a WebSocket middleware is attached. Needs release-note treatment. Fix if anyone hits it is an operator-configured cap, which is additive - Trust assumption.
fail_closedguarantees traffic is not forwarded when inspection fails. It does not guarantee every message is inspected, because a service may decline at preflight or allow a subprotocol it cannot parse. This needs to be stated plainly in the docs - Security exposure pending feat: authenticate OpenShell extension service connections #2430. Until peer authentication lands, this carries complete session content in both directions, in plaintext, over an unauthenticated channel, for the lifetime of every inspected session — materially larger than the shipped HTTP exposure because of volume, the new return path, and compression stripping. Treating feat: authenticate OpenShell extension service connections #2430 as a GA blocker for remote WebSocket middleware, not a blocker for shipping
- Prerequisite. fix(supervisor-middleware): configure HTTP/2 keepalive on the middleware gRPC channel #2474 (HTTP/2 keepalive on the middleware channel) should land first. Pre-existing defect, currently bounded because tonic self-heals per request, but session-fatal once long-lived streams exist
Documentation Impact
rfc/0009-supervisor-middleware/— phase naming and the WebSocket contractarchitecture/sandbox.md,architecture/security-policy.md,architecture/gateway.mddocs/extensibility/supervisor-middleware.mdx,docs/reference/policy-schema.mdx,docs/sandboxes/policies.mdx,docs/security/best-practices.mdx- No
docs/reference/gateway-config.mdxchange — no gateway TOML keys, driver options, or defaults change - No SELinux or AppArmor impact — no change to process identity, execution,
/procaccess, or inter-process visibility - Skill review per the
sync-agent-inframaintenance map:generate-sandbox-policy,openshell-cli,debug-openshell-cluster
Revision 1 — initial plan
PR1 is implemented in #2477.
Highlights:
- Added the bidirectional WebSocket middleware contract and exact forward binding.
- Added preflight, ordered text-message transforms/denials, limits, failure policy, shared admission, and safe OCSF telemetry.
- Added a real tonic sample middleware that redacts an OpenAI
response.createevent; the relay test proves the upstream never receives the original secret-bearing payload. - Preserved raw relay for no-middleware/all-skip sessions and kept return-path/binary inspection deferred to PR2.
Validation completed:
mise run pre-commitmise run cimise run e2e:websocket-conformancemise run e2e:docker
This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Problem Statement
Supervisor middleware can inspect the HTTP request that upgrades a connection to WebSocket, but it cannot inspect messages exchanged after the upgrade. This leaves a gap for applications that carry sensitive or governed content in post-upgrade messages.
Subscription Codex uses WebSocket mode and sends prompts in post-upgrade messages. OpenShell can parse client-to-upstream text messages for built-in WebSocket policy and credential rewriting, but the current supervisor middleware contract exposes only
HTTP_REQUEST/PRE_CREDENTIALS. Privacy Guard and other operator-run middleware therefore cannot inspect, redact, block, or annotate those prompts.The same capability is needed by #2166 for realtime media carried in WebSocket text or binary messages. That issue also covers gRPC and broader media governance; this issue is limited to WebSocket message-level supervisor middleware.
Proposed Design
Add typed post-upgrade WebSocket message operations to supervisor middleware, corresponding to the
WebSocketMessage/before_forwardandWebSocketMessage/before_returnhooks outlined by RFC 0009.The middleware boundary should operate on complete logical messages, not raw RFC 6455 frames:
The implementation may use a bidirectional streaming gRPC operation so one middleware invocation can preserve session context, ordering, cancellation, and backpressure across messages. The exact protobuf shape should be finalized with the implementation, but it must keep WebSocket session processing distinct from the one-shot HTTP upgrade request.
Alternatives Considered
Inspect only the HTTP upgrade
The upgrade request does not contain prompts, media messages, or other application data sent after
101 Switching Protocols, so upgrade admission cannot govern message content.Rely only on built-in WebSocket L7 policy
The existing policy path can evaluate supported client text messages, but it does not provide the external middleware contract, transformations, findings, or binary and return-path coverage needed by Privacy Guard and realtime media use cases.
Expose raw WebSocket frames to middleware
This leaks protocol mechanics into every middleware implementation and makes fragmentation, compression, control frames, and partial disclosure inconsistent. OpenShell should expose reconstructed logical messages and retain ownership of wire correctness.
Use opaque L4 passthrough
L4 passthrough carries bytes but cannot inspect, transform, deny, rate-limit, or audit individual WebSocket messages.
Agent Investigation
HTTP_REQUEST/PRE_CREDENTIALSin v1 and identifies futureWebSocketMessage/before_forwardandWebSocketMessage/before_returnphases.Definition of Done
Checklist