Filtered Parquet scans issue additional object-store requests after ParquetPushDecoder migration

[sesteves]

Describe the bug

After PR #20839 replaced the pull-based Parquet reader with ParquetPushDecoder, a controlled comparison found substantially more object-store requests for a large filtered Parquet scan.

The leading explanation is that predicate and projected-column ranges are discovered in separate NeedsData rounds. DataFusion can coalesce ranges within one get_byte_ranges call, but not with ranges discovered in a later decoder round. Current main still follows this sequence:

try_next_reader()
  -> NeedsData(ranges)
  -> get_byte_ranges(ranges).await
  -> push_ranges(...)
  -> repeat

In architectures where compute and storage are decoupled, each additional object-store round trip adds directly to scan latency. This is especially significant when scans touch many small Parquet files.

The request-level attribution to predicate versus projection rounds is inferred from the source and the A/B results below. The additional requests themselves are directly measured.

To reproduce

Compare a selective filter with a narrow projection over many remote Parquet files while instrumenting AsyncFileReader request counts. Our controlled comparison upgraded only DataFusion from 53.1.0 to 54.1.0 while retaining parquet 58.3.0 and object_store 0.13.2:

Configuration	Object-store requests	Bytes read
DataFusion 53.1.0	708K-709K	~139 GB
DataFusion 54.1.0 with filter pushdown	894K-898K	~86.6 GB
DataFusion 54.1.0 without filter pushdown	583K	~186.3 GB

The DF54 request increase closely matched one additional request per filtered file.

Disabling filter pushdown is not a viable workaround. It reduces request count but more than doubles bytes read and produced a 164-second runtime in this workload.

Expected behavior

Retain the byte savings from predicate pushdown without requiring a separate network round trip for predicate and projected-column ranges.

Possible directions include:

• Coalesce ranges across consecutive decoder rounds.
• Speculatively fetch projected-column ranges where appropriate.
• Extend the arrow-rs decoder API to expose upcoming optional ranges.
• Add request-count coverage using a counting, delayed object store.

Additional context

• PR Use ParquetPushDecoder in ParquetOpener #20839 explicitly identified coalescing and prefetching as goals of the push-decoder migration.
• Closed PR #21370 coalesced adjacent ranges within one NeedsData result, but did not address ranges discovered across separate rounds.
• A search did not find an existing DataFusion issue covering this cross-round request amplification.