Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge SortMergeJoin filtered batches into larger batches #14160

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

comphead
Copy link
Contributor

Which issue does this PR close?

Closes #14050.

Rationale for this change

Filtered SortMergeJoin outputs the data after left row shift which is not performant, merging batches into bigger chunks close to batch_size

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the physical-expr Physical Expressions label Jan 17, 2025
@comphead comphead marked this pull request as draft January 17, 2025 01:17
@comphead comphead marked this pull request as ready for review January 17, 2025 18:36
@comphead comphead changed the title Merge SortMergeJoin filtered batches into bigger batches Merge SortMergeJoin filtered batches into larger batches Jan 17, 2025
@ozankabak
Copy link
Contributor

Can we use the BatchCoalescer and BatchSplitter objects in SMJ as well? AFAICT other joins use such common mechanisms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize filtered SortMergeJoin to avoid producing small/empty batches
2 participants