-
Notifications
You must be signed in to change notification settings - Fork 16
Open
Description
Currently there is a builder function in plan builders called project. It takes a list of expressions as input and generates a ProjectRel that emits only those expressions rather than simply adding those expressions to the plan. In other words, it acts like a select, not like a withColumn transformation, This seemed like a good idea initially, but in retrospect using the same lingo (project) as substrait spec itself, but having the opposite behavior will lead to nothing but trouble down the road.
I'm proposing to instead introduce two functions that both build ProjectRel:
project- this will act like awithColumntransform similar to an out-of-the-box substrait ProjectRel. This will be a "canonical" builder.select- this will be an additional builder that will act like the currentprojectfunction. It will inject an emit message inside a ProjectRel and discard existing fields, emitting only the fields specified by the user.
Metadata
Metadata
Assignees
Labels
No labels