Skip to content

HIVE-29688: IndexOutOfBoundsException when WHERE clause contains IN/EXISTS subqueries AND correlated conditions#6572

Merged
zabetak merged 1 commit into
apache:masterfrom
rubenada:HIVE-29688
Jul 1, 2026
Merged

HIVE-29688: IndexOutOfBoundsException when WHERE clause contains IN/EXISTS subqueries AND correlated conditions#6572
zabetak merged 1 commit into
apache:masterfrom
rubenada:HIVE-29688

Conversation

@rubenada

@rubenada rubenada commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Avoid the problem by forbidding HiveFilterJoinRule to push down predicates containing correlating variables.

Why are the changes needed?

This avoids an IndexOutOfBoundsException while decorrelating this type of plans.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

New q files.

@rubenada

rubenada commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

CI error seems unrelated to the PR, looks like this known issue: https://issues.apache.org/jira/browse/HIVE-29684

zabetak
zabetak approved these changes Jun 30, 2026
View reviewed changes

@zabetak zabetak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Left some minor test improvements.

Comment thread ql/src/test/queries/clientpositive/decorrelate_semi_join.q Outdated
Comment thread ql/src/test/queries/clientpositive/decorrelate_semi_join.q Outdated
Comment thread ql/src/test/queries/clientpositive/decorrelate_semi_join.q Outdated
@rubenada

rubenada commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review @zabetak . I have refactored both tests (which showed different exceptions, both fixed with this patch) on the same file, using real data from part table.

@rubenada rubenada changed the title HIVE-29688: HiveRelDecorrelator does not support semi-joins with correlation variables only in LHS HIVE-29688: IndexOutOfBoundsException when WHERE clause contains IN/EXISTS subqueries AND correlated conditions Jun 30, 2026
@sonarqubecloud

sonarqubecloud Bot commented Jun 30, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@zabetak

zabetak commented Jul 1, 2026

Copy link
Copy Markdown
Member

The last CI run (https://ci.hive.apache.org/job/hive-precommit/job/PR-6572/4/) had an OOM error in TestMiniLlapCliDriver.testCliDriver[llap_io_cache]. The OOM in LLAP is rather frequent these days in CI so I will not relaunch the tests. The failure is unrelated to this PR.

@zabetak zabetak merged commit 8876c17 into apache:master Jul 1, 2026
3 of 4 checks passed
@zabetak zabetak mentioned this pull request Jul 1, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zabetak zabetak zabetak approved these changes

Assignees

No one assigned

Labels

tests unstable

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rubenada @zabetak @asf-ci-hive