Skip to content

Conversation

@jan-auer
Copy link
Member

@jan-auer jan-auer commented Oct 15, 2025

This is an attempt to allow ORDER BY while using RangeQuerySetWrapper. This allows to iterate more efficiently when filtering large datasets with existing indexes.

We still require a unique column at the very end to ensure consistent iteration, usually the PK. We also do support min_value and result_value_getter but only on the unique column.

This uses tuple comparison to create a concise filter for the cursor queries, which also works if it includes nullable columns. Since Django I could not find native support for tuples in Django ORM, this uses a workaround with a custom Tuple type based on Func to reuse as much serialization logic as possible, along with a manual Lookup expression to render the comparison. This creates queries like this:

WHERE (x, y, id) >= ($x, $y, $id)
ORDER BY x, y, id

The alternative would be unrolling the tuple comparsion. However, I've read that this often leads to suboptimal use of existing indexes in postgres. Apart from that, it would create rather large expressions that grow with the number of fields:

WHERE (x > $x) 
   OR (x = $x AND y > $y)
   OR (x = $x AND y = $y AND id >= $id)
ORDER BY x, y, id

Ref FS-105

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 15, 2025
@jan-auer jan-auer requested a review from wedamija October 15, 2025 11:10
@linear
Copy link

linear bot commented Oct 15, 2025

@getsantry
Copy link
Contributor

getsantry bot commented Nov 6, 2025

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!


"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

@getsantry getsantry bot added the Stale label Nov 6, 2025
@getsantry getsantry bot closed this Nov 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants