Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add StrictMetricsEvaluator #963

Merged
merged 21 commits into from
Feb 26, 2025

Conversation

jonathanc-n
Copy link
Contributor

@jonathanc-n jonathanc-n commented Feb 12, 2025

part of #735. Added StrictMetricsEvaluator.

@jonathanc-n
Copy link
Contributor Author

Tests will be added tomorrow.

@jonathanc-n
Copy link
Contributor Author

Tests have been added, this implementation follows the pyiceberg implementation. cc @Fokko @liurenjie1024 @ZENOTME

Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@viirya Thanks for working on this, and also copying over the tests 🙌 I've left some comments

return ROWS_MUST_MATCH;
}

let mut evaluator = Self::new(data_file, schema);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still learning rust, why is this one mutable?

Copy link
Contributor Author

@jonathanc-n jonathanc-n Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bound predicate visitor takes in a mutable reference of the evaluator. in rust you have to declare as mutable as well as pass it in with mut to let the compiler know the function can modify

@jonathanc-n
Copy link
Contributor Author

@Fokko Thanks for the review. I got a bit mixed up with the logic with only nulls vs may contain null 😆. Should be good now

Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n for this great pr, and all the tests! Just some minor suggestions, others looks great!

data_file: &'a DataFile,
}

impl<'a> StrictMetricsEvaluator<'a> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it has a lot of duplication with InclusiveMetricsEvaluator, maybe we could unify them in future.

@jonathanc-n
Copy link
Contributor Author

@liurenjie1024 @Fokko Thank you for the reviews, problems should be resolved.

Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonathanc-n for this pr!

@liurenjie1024 liurenjie1024 merged commit eb4c668 into apache:main Feb 26, 2025
17 checks passed
@jonathanc-n jonathanc-n deleted the add-strict-metrics-evaluator branch March 28, 2025 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants