Skip to content

Cherry-pick apache/datafusion#27547: preserve date_trunc ordering for timezone-aware timestamps at minute and finer granularities - #94

Merged
xudong963 merged 1 commit into
branch-56from
xudong963/cherry-pick-27547-date-trunc-ordering
Nov 19, 2026
Merged

xudong963 merged 1 commit into
branch-56from
xudong963/cherry-pick-27547-date-trunc-ordering

Conversation

@xudong963

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

apache#25409 stopped date_trunc from propagating input ordering for every timezone-aware timestamp, because truncating in local time can go backwards across a timezone transition. For minute and finer granularities, date_trunc never consults the timezone. It rounds the epoch value down, so it cannot reorder values.

What changes are included in this PR?

git cherry-pick -x 5a30a391 applied cleanly with no adaptations. DateTruncFunc::output_ordering now keeps the input ordering for a timezone-aware timestamp when the precision is a known constant and truncates_in_input_unit holds, which is the same predicate that selects the arithmetic kernel. hour and coarser granularities in a timezone stay unordered.

For Atlas, the 22 minute/second aggregate MVs (crypto, forex, indices, stocks sip/otc, futures) should get the time-bucket key back. The 10 day-granularity ones stay as they are; their leading date key already provides most of the streaming.

Are these changes tested?

  • cargo test -p datafusion-functions --lib date_trunc: 16 passed, including the new output_ordering_with_timezone_depends_on_granularity.
  • The new timestamps.slt cases pass, along with every other .slt file that mentions date_trunc (14 files: group_by, group, order, sort_pushdown, range_sorted_time_bin_agg, information_schema, clickbench, datetime/*, pg_compat_timestamptz, spark/datetime/*).
  • cargo clippy -p datafusion-functions --all-targets --all-features -- -D warnings and cargo fmt --check are clean.

Are there any user-facing changes?

Queries that sort or group timezone-aware timestamps by date_trunc at minute or finer granularity, over input ordered on the timestamp, no longer add a sort and use clustered aggregation. Results are unchanged.

…inute and finer granularities

For minute and finer granularities date_trunc rounds the epoch value down
without consulting the timezone, so it cannot reorder timestamps across a
timezone transition. Propagate the input ordering in that case, using the
predicate that selects the arithmetic kernel.

(cherry picked from commit 5a30a39)
@xudong963
xudong963 merged commit 775d606 into branch-56 Nov 19, 2026
73 checks passed
@xudong963
xudong963 deleted the xudong963/cherry-pick-27547-date-trunc-ordering branch November 19, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants