Skip to content

fix(proto-common): return an error instead of panicking on bad input - #27056

Open
geoHeil wants to merge 2 commits into
apache:mainfrom
geoHeil:proto-common-decode-errors
Open

geoHeil wants to merge 2 commits into
apache:mainfrom
geoHeil:proto-common-decode-errors

Conversation

@geoHeil

@geoHeil geoHeil commented Nov 3, 2026 •

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

A process can decode protobuf plans and values that it did not encode itself. A distributed worker gets them from a coordinator, and a service gets them from a client. In that setting, a malformed message is input and should fail with an error. Today four sites in datafusion-proto-common's from_proto panic on such input. A process built with panic = "abort" cannot catch that panic, so one bad message takes the whole process down instead of failing one request.

Decoding a protobuf message that this process did not encode could panic
at four sites in from_proto. Each now returns an error:

- a decimal ScalarValue whose byte string has the wrong length
- a nested ScalarValue whose record batch has no column, or a column of a
  different type than its kind
- a Union type with more than 128 children and no explicit type ids
- CsvOptions with an empty delimiter or quote

For apache#27004.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kosiew kosiew left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@geoHeil, thanks for the fix. The fallible conversions correctly reject malformed protobuf inputs without changing valid decoding behavior.

The regression tests cover all four panic sites. I have two non-blocking suggestions to strengthen test reliability and coverage.

);
}

let no_columns = crate::protobuf_common::ScalarNestedValue {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use matching IPC metadata and payload from a genuine zero-column batch? The current fixture replaces the schema but retains list IPC data, relying on Arrow ignoring the extra field nodes and buffers.

};

assert!(matches!(
DataType::try_from(&union(128)).unwrap(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider asserting that fields f0 through f127 retain implicit IDs 0..=127. The current length assertion would miss reordered children or incorrect ID assignments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

proto Related to proto crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

proto-common: decoding a malformed scalar, union type or CSV options panics instead of erroring

3 participants