Skip to content

Cap RPC payload logging with size-limited proto summaries#1668

Merged
milos-lk merged 2 commits into
mainfrom
milos/cap-rpc-payload-logging
Jul 15, 2026
Merged

Cap RPC payload logging with size-limited proto summaries#1668
milos-lk merged 2 commits into
mainfrom
milos/cap-rpc-payload-logging

Conversation

@milos-lk

Copy link
Copy Markdown
Contributor

Problem

The psrpc logging interceptors marshal full request/response protos into every access log line. Payloads of unbounded size — e.g. a ListEgressResponse page with 100+ items (~500KB JSON) — produce log lines that exceed the logging backend's per-message limits. The collector splits them into truncated chunks that fail JSON parsing, so they land unstructured with the wrong status, polluting the error stream (and past hard ingestion caps the remainder is silently discarded).

Change

  • New logger.ProtoWithLimit(msg, maxBytes): behaves like logger.Proto, but when the message's wire size exceeds maxBytes it logs a compact summary instead of the full contents:
    • message type (truncatedProto) and wire size (protoBytes)
    • top-level scalar fields preserved — sensitivity redaction still applies, strings over 256 bytes capped with a size marker
    • repeated/map fields reduced to <field>Count
    • nested messages dropped
  • The size check runs inside MarshalLogObject, so it costs nothing when the log level is disabled.
  • psrpc logging interceptors (client/server RPC, stream, multirpc) now use it with a 32KB wire-size cap. Wire→JSON expansion is roughly 2–4× for typical payloads, keeping encoded lines well within collector limits.

Example: a truncated ListEgressResponse now logs as

"response":{"truncatedProto":"livekit.ListEgressResponse","protoBytes":412873,"itemsCount":103}

Proto/UnredactedProto and all other call sites are unchanged.

🤖 Generated with Claude Code

psrpc access logs marshal full request/response protos. Unbounded
payloads (e.g. large ListEgressResponse pages) produce log lines that
exceed collector line limits, getting split into unparseable chunks.

Add logger.ProtoWithLimit: when a message's wire size exceeds the
limit, log a summary instead — message type, wire size, top-level
scalars (sensitivity redaction applied, long strings capped), counts
for repeated/map fields. Use it in the psrpc logging interceptors with
a 32KB cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: bf82616

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

zapcore.MapObjectEncoder stores AddInt values as int, not int64.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@milos-lk milos-lk merged commit acc2748 into main Jul 15, 2026
9 checks passed
@milos-lk milos-lk deleted the milos/cap-rpc-payload-logging branch July 15, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants