Restore mergeAndSetValidity as a single ColumnView API - #23919
Conversation
Signed-off-by: Rishi Chandra <rishic@nvidia.com>
|
/ok to test aeb0926 |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. 📝 SummarySummary by CodeRabbit
WalkthroughChangesValidity merge API cleanup
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: ⚪ Minimal · up to This change restores the validity-merge API on ColumnView to avoid unnecessary copies for callers holding views. No current merge-blocking risk is identified. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
| * If applying the null mask would be a no-op and this is a {@link ColumnVector}, the original | ||
| * column is returned with incremented refcount. Otherwise, a deep copy of the column is made. | ||
| * For a non-owning ColumnView, a deep copy must be made in either case. |
There was a problem hiding this comment.
I'm still not clear and don't see how the original column returns no-op incRefCount (to output ColumnVector) if calling from a ColumnView object?
There was a problem hiding this comment.
It does not. As the docstring states if we have a ColumnView we have to do a deep copy to produce an owning result. (This would be row 3 of the table in the PR desc). Only if we have a ColumnVector do we have the no-op incRefCount.
There was a problem hiding this comment.
But this PR is removing the ColumnVector overload thus the input is going to always be a ColumnView.
There was a problem hiding this comment.
It removes the ColumnVector override, but ColumnVector extends ColumnView. So this will work:
ColumnVector cv = ...;
cv.mergeAndSetValidity(...);and if the above is a no-op, we invoke copyToColumnVector(), which ColumnVector overrides to be just an incRefCount() and is thus zero copy.
There was a problem hiding this comment.
copyToColumnVector(), which ColumnVector overrides to be just an incRefCount()
This is the main point. Everything is explained by it. Yay!
Please add more comment on the docs clarifying this -- it is useful to understand what is going on.
There was a problem hiding this comment.
👍 added a comment to clarify.
|
/ok to test ec35f42 |
Description
Supersedes #23249. This reverts the move of mergeAndSetValidity into ColumnVector.
By keeping the API on ColumnView, this ensures that for a caller holding a ColumnView, they may use this API without incurring an additional copy in the event that the mask changes (the call is not a no-op). I.e. consider this table comparing the placement of the API and the copies incurred:
An example of the 4th case is NVIDIA/cudf-spark#14846.
Checklist