Repository navigation
Conversation
Decoding a protobuf message that this process did not encode could panic at four sites in from_proto. Each now returns an error: - a decimal ScalarValue whose byte string has the wrong length - a nested ScalarValue whose record batch has no column, or a column of a different type than its kind - a Union type with more than 128 children and no explicit type ids - CsvOptions with an empty delimiter or quote For apache#27004. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
@geoHeil, thanks for the fix. The fallible conversions correctly reject malformed protobuf inputs without changing valid decoding behavior.
The regression tests cover all four panic sites. I have two non-blocking suggestions to strengthen test reliability and coverage.
| ); | ||
| } | ||
|
|
||
| let no_columns = crate::protobuf_common::ScalarNestedValue { |
There was a problem hiding this comment.
Could we use matching IPC metadata and payload from a genuine zero-column batch? The current fixture replaces the schema but retains list IPC data, relying on Arrow ignoring the extra field nodes and buffers.
| }; | ||
|
|
||
| assert!(matches!( | ||
| DataType::try_from(&union(128)).unwrap(), |
There was a problem hiding this comment.
Consider asserting that fields f0 through f127 retain implicit IDs 0..=127. The current length assertion would miss reordered children or incorrect ID assignments.
Which issue does this PR close?
Rationale for this change
A process can decode protobuf plans and values that it did not encode itself. A distributed worker gets them from a coordinator, and a service gets them from a client. In that setting, a malformed message is input and should fail with an error. Today four sites in
datafusion-proto-common'sfrom_protopanic on such input. A process built withpanic = "abort"cannot catch that panic, so one bad message takes the whole process down instead of failing one request.