Skip to content

feat: multiple builders for ProjectRel #124

@tokoko

Description

@tokoko

Currently there is a builder function in plan builders called project. It takes a list of expressions as input and generates a ProjectRel that emits only those expressions rather than simply adding those expressions to the plan. In other words, it acts like a select, not like a withColumn transformation, This seemed like a good idea initially, but in retrospect using the same lingo (project) as substrait spec itself, but having the opposite behavior will lead to nothing but trouble down the road.

I'm proposing to instead introduce two functions that both build ProjectRel:

  • project - this will act like a withColumn transform similar to an out-of-the-box substrait ProjectRel. This will be a "canonical" builder.
  • select - this will be an additional builder that will act like the current project function. It will inject an emit message inside a ProjectRel and discard existing fields, emitting only the fields specified by the user.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions