WIP prefetch - #23901

Draft

Matt711 wants to merge 27 commits into

NVIDIA:mainfrom

Matt711:fea/polars/prefetch-byte-ranges

Commits on Aug 31, 2026

Commits on Sep 1, 2026

add more nvtx annotations
Matt711
committed
4b3a609
Merge branch 'main' into fea/polars/prefetch-byte-ranges
Matt711
authored
f31dd23
simplify unnecessary configs
Matt711
committed
340e4ee
move more work off the event loop threat
Matt711
committed
f6378c0
move prefetch to a separate threadpool
Matt711
committed
4fb7fd0
split out coalesce logic from prange submits
Matt711
committed
46196c4
move prepare prefetch back to pyexecutor threadpool
Matt711
committed
51ecb2a
use the py executor thread pool
Matt711
committed
7633677
switch back to separate pools
Matt711
committed
9fd1b83
hybrid scan eligibility
Matt711
committed
01e7bc1
add ordered vs race modes and hold write lock on rapidsmpf::Beffer for longer
Matt711
committed
3ed547f
decouple prefectch executor max cap from max concourrent io tasks
Matt711
committed
2894259
less event loop thread hops
Matt711
committed
703fa7a

Commits on Sep 2, 2026

use dedicated envent loop thread for prefetch things
Matt711
committed
cdcf3fc
move back to main event loop thread
Matt711
committed
858fb45
add query prefetch pipeline
Matt711
committed
0f8d38b
compare with 23317
Matt711
committed
ae0035b
add batch moded
Matt711
committed
75c2eea
add pace mode
Matt711
committed
7a6ba9c
fix deadloack
Matt711
committed
b659218
add more fifo queues
Matt711
committed
b1fa753
turn preftching off/on
Matt711
committed
cbf3008