Add to MULTI_POLL backend the execution-time dispatch pattern SHARED_QUEUE

[kingcrimsontianyu]

This PR has 2 major parts:

• A third dispatch mode KVIKIO_REMOTE_IO_REACTOR_DISPATCH=SHARED_QUEUE is added to the MULTI_POLL backend. In this mode, sub-ranges wait in one pool-wide queue, and a reactor pulls one only after reserving a concurrency slot for it. This way the work binds to a reactor at execution time instead of at submission time. The mode requires a non-zero KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS and falls back to PER_CHUNK without one. This mode improves TPC-H performance for both Polars and Presto, shown in the section below Add to MULTI_POLL backend the execution-time dispatch pattern SHARED_QUEUE #1062 (comment).

• The reactor loop has now become quite complex, as it incorporates several key logic in a flat pattern, such as concurrency restriction, device copy, retry mechanism, shared queue dispatch, etc. This PR completely reorganizes the reactor loop for improved readability. Comments are modified by AI agent for better clarity.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[kingcrimsontianyu]

/ok to test

[kingcrimsontianyu]

/ok to test

[kingcrimsontianyu]

Performance result

• TPC-H SF-1K
• AWS EC2 g7e.12xlarge instance

cudf-polars

- 64 MiB task size
- Arguments:
  --frontend spmd
  --max-concurrent-io-tasks=8 
  --pinned-memory
  --pinned-initial-pool-size 68719476736 --rapidsmpf-statistics
  --no-print-results --no-collect-traces --no-summarize all
+-----------------+----------------------+--------+--------+------------+------------+
|     backend     |    configuration     | iter-1 | iter-2 | i1 vs easy | i2 vs easy |
+-----------------+----------------------+--------+--------+------------+------------+
| easy_threadpool | 384 threads          |  159.6 |  137.3 |      +0.0% |      +0.0% |
| SHARED_QUEUE    | 24 threads, conn 384 |  153.4 |  137.7 |      -3.9% |      +0.3% |
| PER_CHUNK       | 24 threads, conn 384 |  167.6 |  151.5 |      +5.0% |     +10.3% |
| PER_PREAD       | 24 threads, conn 384 |  166.0 |  150.2 |      +4.0% |      +9.4% |
+-----------------+----------------------+--------+--------+------------+------------+

Presto GPU

- WXD-launch velox and presto branch
- This KvikIO branch
- 2 max drivers
- driver.max-split-preload=6
- 64 MiB task size
+-----------------+----------------------+--------+--------+------------+------------+
|     backend     |    configuration     | iter-1 | iter-2 | i1 vs easy | i2 vs easy |
+-----------------+----------------------+--------+--------+------------+------------+
| easy_threadpool | 384 threads          |  120.2 |  109.4 |      +0.0% |      +0.0% |
| SHARED_QUEUE    | 48 threads, conn 384 |  121.6 |  109.6 |      +1.2% |      +0.2% |
| PER_CHUNK       | 48 threads, conn 384 |  125.7 |  113.5 |      +4.6% |      +3.8% |
| PER_PREAD       | 48 threads, conn 384 |  126.1 |  113.1 |      +4.9% |      +3.4% |
+-----------------+----------------------+--------+--------+------------+------------+

[kingcrimsontianyu]

This function is the key part of this PR. Under the SHARED_QUEUE mode, the work assignment is now done at execution-time instead of submission-time. The queue wants to hand a total of share=queue-size / reactor-count (rounded up) works to each reactor, and the actual number of works accepted by a reactor depends collectively on the value of share, the reactor-private concurrency cap, and the per-(reactor, cuda context) bounce buffer cap.