Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Search
/
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
NVIDIA
/
cudf
Public
Notifications
You must be signed in to change notification settings
Fork
1.1k
Star
9.8k
Code
Issues
1.2k
Pull requests
170
Discussions
Actions
Projects
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Discussions
Actions
Projects
Security and quality
Insights
Support hive partitioned parquet files (except HybridScan) in cudf-polars
- #23966
#23966
Open
mroeschke
wants to merge 52 commits into
NVIDIA:main
NVIDIA/cudf:main
from
mroeschke:cudf_polars/feat/hive_partitioning
mroeschke/cudf:cudf_polars/feat/hive_partitioning
Copy head branch name to clipboard
Conversation
Commits
52
(52)
Checks
Files changed
Open
Support hive partitioned parquet files (except HybridScan) in cudf-polars
#23966
mroeschke
wants to merge 52 commits into
NVIDIA:main
NVIDIA/cudf:main
from
mroeschke:cudf_polars/feat/hive_partitioning
mroeschke/cudf:cudf_polars/feat/hive_partitioning
Copy head branch name to clipboard
Commits
Commits on Sep 1, 2026
Add HivePartitions container for hive-partitioned scans
Show description for ab49ec4
mroeschke
committed
ab49ec4
View commit details
Copy full SHA for ab49ec4
Browse repository at this point
Materialize hive columns in the parquet Scan
Show description for 599e257
mroeschke
committed
599e257
View commit details
Copy full SHA for 599e257
Browse repository at this point
Handle hive scans that read no columns from the files
Show description for 9b45b33
mroeschke
committed
9b45b33
View commit details
Copy full SHA for 9b45b33
Browse repository at this point
Enable hive-partitioned parquet scans
Show description for 9973ac5
mroeschke
committed
9973ac5
View commit details
Copy full SHA for 9973ac5
Browse repository at this point
Support hive partitioning in the streaming scans
Show description for 57800ce
mroeschke
committed
57800ce
View commit details
Copy full SHA for 57800ce
Browse repository at this point
Fix include_file_paths under a pushed-down filter
Show description for d5e2263
mroeschke
committed
d5e2263
View commit details
Copy full SHA for d5e2263
Browse repository at this point
Round out hive scan coverage and fix stale xfail reasons
Show description for 8388688
mroeschke
committed
8388688
View commit details
Copy full SHA for 8388688
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
c7e4bba
View commit details
Copy full SHA for c7e4bba
Browse repository at this point
Commits on Sep 2, 2026
Drop the unreachable hive path-count assertion
Show description for b51b42c
mroeschke
committed
b51b42c
View commit details
Copy full SHA for b51b42c
Browse repository at this point
Trim the rows_per_path parameter description
mroeschke
committed
143a445
View commit details
Copy full SHA for 143a445
Browse repository at this point
Document the parquet scan helpers in numpydoc style
Show description for b7167d1
mroeschke
committed
b7167d1
View commit details
Copy full SHA for b7167d1
Browse repository at this point
Stop walking paths once the row budget is spent
Show description for 61781af
mroeschke
committed
61781af
View commit details
Copy full SHA for 61781af
Browse repository at this point
Hold the hive partition values as the polars frame
Show description for f2ddf50
mroeschke
committed
f2ddf50
View commit details
Copy full SHA for f2ddf50
Browse repository at this point
Build the hive streaming IR from the engine fixture
Show description for b9e9ace
mroeschke
committed
b9e9ace
View commit details
Copy full SHA for b9e9ace
Browse repository at this point
Drive the per-path row counts through scan_parquet
Show description for babd2b5
mroeschke
committed
babd2b5
View commit details
Copy full SHA for babd2b5
Browse repository at this point
Commits on Sep 3, 2026
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
238bc84
View commit details
Copy full SHA for 238bc84
Browse repository at this point
Remove helper functions
mroeschke
committed
85d32a7
View commit details
Copy full SHA for 85d32a7
Browse repository at this point
Fix the rewritten HivePartitions identity methods
Show description for 01f39b3
mroeschke
committed
01f39b3
View commit details
Copy full SHA for 01f39b3
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
868f1a7
View commit details
Copy full SHA for 868f1a7
Browse repository at this point
Generalize HivePartitions into PerPathValues and reuse it for file paths
Show description for f054b99
mroeschke
committed
f054b99
View commit details
Copy full SHA for f054b99
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
86abc95
View commit details
Copy full SHA for 86abc95
Browse repository at this point
Prune AI comments
mroeschke
committed
b0f036e
View commit details
Copy full SHA for b0f036e
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
6b7ba47
View commit details
Copy full SHA for 6b7ba47
Browse repository at this point
Remove another comment
mroeschke
committed
a6fa1f6
View commit details
Copy full SHA for a6fa1f6
Browse repository at this point
Commits on Sep 4, 2026
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
7d367d3
View commit details
Copy full SHA for 7d367d3
Browse repository at this point
Skip test_from_polars_zero_width for Polars less than 1.38
mroeschke
committed
022b151
View commit details
Copy full SHA for 022b151
Browse repository at this point
Add expected failure for test_write_delta
mroeschke
committed
3f5225b
View commit details
Copy full SHA for 3f5225b
Browse repository at this point
Add another expected failure
mroeschke
committed
79f9cb1
View commit details
Copy full SHA for 79f9cb1
Browse repository at this point
Commits on Sep 14, 2026
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
3ab5e2d
View commit details
Copy full SHA for 3ab5e2d
Browse repository at this point
Commits on Sep 15, 2026
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
985e6fe
View commit details
Copy full SHA for 985e6fe
Browse repository at this point
Create and use Scan.slice_hive_parts instead
mroeschke
committed
721b7a1
View commit details
Copy full SHA for 721b7a1
Browse repository at this point
Remove xpassing Polars tests, add test coverage
mroeschke
committed
d318f00
View commit details
Copy full SHA for d318f00
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
7fb1332
View commit details
Copy full SHA for 7fb1332
Browse repository at this point
Remove xpassing Polars tests
mroeschke
committed
99b3455
View commit details
Copy full SHA for 99b3455
Browse repository at this point
Commits on Sep 17, 2026
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
ea9cb8b
View commit details
Copy full SHA for ea9cb8b
Browse repository at this point
Correct df.width 0 comment, clarify broadcast condidtion
mroeschke
committed
eedc067
View commit details
Copy full SHA for eedc067
Browse repository at this point
Document stream validity of source_index on gather
mroeschke
committed
55d44bf
View commit details
Copy full SHA for 55d44bf
Browse repository at this point
Document stream ordering requirement of add_file_paths
mroeschke
committed
a537c59
View commit details
Copy full SHA for a537c59
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
359c6a8
View commit details
Copy full SHA for 359c6a8
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
e056db0
View commit details
Copy full SHA for e056db0
Browse repository at this point
Make PerPathValues members without external callers private
Show description for 83f75a5
mroeschke
committed
83f75a5
View commit details
Copy full SHA for 83f75a5
Browse repository at this point
Annotate test_is_uniform parameters
mroeschke
committed
8cff05d
View commit details
Copy full SHA for 8cff05d
Browse repository at this point
Remove expected_flavor from test_hive_partitioned_streaming_scan
mroeschke
committed
cbb1794
View commit details
Copy full SHA for cbb1794
Browse repository at this point
Use the streaming_engine fixture for streaming-only hive scan tests
mroeschke
committed
33dc067
View commit details
Copy full SHA for 33dc067
Browse repository at this point
Check row order in hive scan tests except for group_by
mroeschke
committed
8639a71
View commit details
Copy full SHA for 8639a71
Browse repository at this point
Merge remote-tracking branch 'upstream/main' into cudf_polars/feat/hive_partitioning
mroeschke
committed
be1a93f
View commit details
Copy full SHA for be1a93f
Browse repository at this point
Commits on Sep 18, 2026
Assert hive partition values have one row per path
mroeschke
committed
44efd5d
View commit details
Copy full SHA for 44efd5d
Browse repository at this point
Assert hive partition values match task paths in ParquetScanTask
mroeschke
committed
9fb87df
View commit details
Copy full SHA for 9fb87df
Browse repository at this point
Create _validate_hive_parts_info and consolidate
mroeschke
committed
cfa744e
View commit details
Copy full SHA for cfa744e
Browse repository at this point
Prune covered per_path tests
mroeschke
committed
ee7460e
View commit details
Copy full SHA for ee7460e
Browse repository at this point
Remove test_per_path to see what would end up being uncovered
mroeschke
committed
8162a3e
View commit details
Copy full SHA for 8162a3e
Browse repository at this point
Add decimal testing, remove some redundant streaming test_scan tests
mroeschke
committed
5d0c4e8
View commit details
Copy full SHA for 5d0c4e8
Browse repository at this point
You can’t perform that action at this time.
