feat(ocsf): emit full JSON records to supervisor stderr

[johntmyers]

Summary

Enabling ocsf_json_enabled now emits full OCSF records to supervisor stderr with an OCSF-JSON marker, so container log collectors can retrieve structured events without accessing the supervisor's private log filesystem. The output works even when file logging is unavailable and respects the existing schema-version setting.

Related Issue

Closes #4322

Changes

• Preserve shorthand output and optional plain JSONL files; add timestamped, compact JSON console records with the original event UID.
• Share a bounded, nonblocking stderr queue across diagnostics, shorthand, and JSON. Submit complete lines to prevent interleaved records; drop new lines when its 1,024-line capacity is full.
• Keep JSON console filtering independent of diagnostic filters and file-sink initialization.
• Document collector framing, delivery limits, and the Docker tmpfs access trap; update relevant operator skills.
• Keep supervisor-to-gateway gRPC log content and CLI/API representations unchanged.

Testing

• OCSF crate tests and supervisor library/binary tests pass, including toggles, downgrade, event identity, concurrent writes, unavailable file storage, disabled diagnostics, and blocked stderr.
• Rust formatting and Clippy checks pass for the affected crates and new E2E test. The repository pre-commit suite also passes, including full-workspace Clippy, the sandbox perf harness, E2E and example lint, license and lockfile checks, Helm, Python, TypeScript, and protobuf validation.
• Supervisor Docker image builds successfully.
• Fern validation and Markdown lint pass (Fern reports three warnings).
• mise run e2e:docker passes all six CLI conformance scenarios and the new collector E2E test against openshell/supervisor:ocsf-json-4322. The test confirms JSON is emitted on stderr after a live settings toggle, shorthand remains available, and gateway log responses contain no JSON records.

Console delivery is best-effort. The stderr queue can drop records under backpressure, and retention remains controlled by the runtime and external collector.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Relevant implementation and operator documentation updated

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4323.docs.buildwithfern.com/openshell

[johntmyers]

gator-agent

Blocked

The independent review identified a security-sensitive concern requiring private maintainer triage under SECURITY.md. Detailed evidence is retained in the sandbox and is not being posted publicly.

Action required: @johntmyers and the maintainers should coordinate private security review through the channels in SECURITY.md before Gator advances to test dispatch.

Gator metadata

• Validation: Maintainer-authored implementation of the collector workflow scoped in feat(ocsf): emit opt-in OCSF-JSON records to supervisor stderr for log collectors #4322.
• Docs: Relevant Fern and operator documentation updated; one non-blocking wording inconsistency noted.
• Checks: DCO and vouch passed; branch checks are running. No test dispatch performed this cycle.
• Head SHA: b78beea3f20f507a5544f722abe6229252026f1c
• Base SHA: 249c6e727ffd6cf3c6171758acf49e18bb022eb4
• Merge base SHA: a4954d3a99f3c578a3cd22c9281c5286a2dfbbdd
• Patch ID: 6ea32083dc3ba18997d260a089ea84a8799681ba
• Gator payload: 11
• Review mode: initial
• Previous reviewed SHA: none
• Review budget exhausted: no
• Maintainer decision required: yes — private security triage
• Next state: gator:blocked
• Blocked reason: private_security_review_required

[github-actions]

Label test:e2e applied for a20525a. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[johntmyers]

gator-agent

PR Review Status

@johntmyers, I checked your latest commit and its regression coverage against the earlier review concern. That finding is resolved, and the independent review found no remaining blocking code findings.

Action required: a maintainer should select Re-run all jobs for Branch E2E Checks run 37707051030. The E2E Label Help bot requires this rerun after applying test:e2e; the sandbox policy denied Gator's authorized rerun request, so E2E has not started.

Blocking findings: None.

Carried findings: None; GATOR-b78beea3-01 is resolved by the current head.

Gator metadata

• Validation: Maintainer-authored implementation of the collector workflow scoped in #4322.
• Docs: Relevant Fern and operator documentation updated; the earlier non-blocking schema wording suggestion remains optional.
• Checks: DCO and Trivy passed; Helm gate is green; Branch Checks is pending. Required checks are not all green.
• E2E: test:e2e applied. The mirror matches the current head, so /ok to test is unnecessary. Current E2E attempt 1 skipped its suites; the bot requires a rerun. REST rerun was rejected by sandbox policy, and no new attempt was queued.
• Review validation: Code-only independent review; local tests were not executed this cycle.
• Head SHA: a20525aabf8e3d0ad901ee6b0c2c415b2aed303b
• Base SHA: 249c6e727ffd6cf3c6171758acf49e18bb022eb4
• Merge base SHA: a4954d3a99f3c578a3cd22c9281c5286a2dfbbdd
• Patch ID: f64e9360c347756cabc8ea1c0c2eac12fb21890a
• Gator payload: 11
• Review mode: initial (authoritative ledger; earlier private evidence retained and reconciled)
• Previous reviewed SHA: none in ledger; prior blocked head b78beea3f20f507a5544f722abe6229252026f1c checked for carried obligations.
• Review budget exhausted: no
• Maintainer decision required: no review convergence decision; test dispatch action remains.
• Next state: gator:blocked
• Blocked reason: test_dispatch_required

[johntmyers]

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: Gator's current-head review recorded no remaining blocking code findings, and the PR reached gator:merge-ready before merging.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.