Skip to content

Expose public API for type string conversion and NamedStruct ↔ ExtensionColumn mapping #98

@wackywendell

Description

@wackywendell

Problem

When implementing Explainable for extension relations that carry a base_schema (which is very common — most ExtensionLeafRel details include a NamedStruct), you need two things:

  1. Convert between Substrait Type protos and the compact type strings used in the text format (e.g. i64, string?, timestamp_tz). substrait-explain already has this mapping internally in both the parser (parser/types.rs) and textify (textify/types.rs) modules, but it's not publicly accessible. Every Explainable implementation that needs to render a schema in output columns currently has to reimplement this mapping.

  2. Convert between NamedStruct and Vec<ExtensionColumn>. A NamedStruct is a list of names paired with a struct of types — this maps directly to the => name:type, name:type output column syntax that ExtensionArgs already supports via ExtensionColumn::Named. But there's no utility to do this conversion, so every wrapper that carries a schema has to write the same loop pairing names with types and converting each type to/from a string.

Request

Expose public APIs for:

  1. A function to convert a substrait::proto::Type to its text-format string representation, and the inverse parse function. Something like:

    pub fn type_to_string(ty: &Type) -> Result<String, ...>
    pub fn string_to_type(s: &str) -> Result<Type, ...>
  2. Convenience functions to convert between NamedStruct and Vec<ExtensionColumn>:

    pub fn schema_to_columns(schema: &NamedStruct) -> Result<Vec<ExtensionColumn>, ...>
    pub fn columns_to_schema(columns: &[ExtensionColumn]) -> Result<NamedStruct, ...>

These would make it straightforward to implement Explainable for any extension that carries a schema, without duplicating substrait-explain's internal type mapping.

Metadata

Metadata

Assignees

No one assigned

    Labels

    datadogNeeded for internal development at Datadog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions