Automating predicate pushdown via declared capabilities

[xonx4l]

What about a 'Capabilities API' for TableProvider? Instead of manually walking Expr trees for pushdown, could providers simply declare supported filters/joins and let the optimizer handle the pruning automatically?

[ryux1]

There is already a capability negotiation point for scan filters, but it is expression-level rather than a static declaration: TableProvider::supports_filters_pushdown(&[&Expr]) returns Exact, Inexact, or Unsupported for each candidate. The logical filter-pushdown rule uses those answers to decide which predicates may move into TableScan.filters and which must remain above the scan.

That dynamic shape is important. Support often depends on more than an operator name: column type, casts, function volatility, nested fields, null semantics, collation, connector version, and combinations such as a = 1 AND unsupported_udf(b). A declaration such as “supports equality and ranges” would still need a shared expression matcher precise enough to encode those constraints, or it risks incorrect Exact claims that remove a necessary residual filter.

A useful Capabilities API could therefore be additive: reusable predicates/builders that implement common expression subsets and produce the existing per-expression TableProviderFilterPushDown results. That would remove repeated tree-walking code without replacing the correctness boundary. Providers with backend-specific semantics could still override the dynamic method.

Join pushdown is a different layer. A TableProvider represents one scan and does not currently own a general join-capability contract. Pushing a join into an external engine needs to establish that both inputs share a compute context and that the engine can preserve the join type, predicates, casts, and output schema. That fits a federation/extension optimizer (such as datafusion-federation) better than a scan-only TableProvider flag.

So the short version is: automatic pruning already exists after providers classify filters; shared declarative helpers could make that classification much easier, but a fully static list is probably not expressive enough, and joins need a plan-level federation capability rather than only table metadata.