Skip solver specific example test when solver not installed #774
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
IgnaceBleukx
left a comment
IgnaceBleukx
left a comment
There was a problem hiding this comment.
Looks good, though I would prefer to change the variable name (see comment)
| if solver == 'minizinc' and any(x in example for x in SKIP_MZN): | ||
| return pytest.skip(reason=f"exclude {example} for {solver}, too slow or solver-specific") | ||
| # Skip solver specific tests for which the solver is not installed | ||
| for skip_name in _SOLVER_SPECIFIC_EXAMPLES_REVERSE.keys(): |
Collaborator
There was a problem hiding this comment.
Hmm I find skip_name unintuitive, maybe just example_name?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some of the examples which are included in our test set target very specific solver backends, different from the default OR-Tools. Whilst not an issue on the GitHub runner (since these solvers get installed in the runner), these tests will fail on a brand new install. I'm of the opinion that all tests should pass with the default minimal install. So I made these tests optional, adding the mechanism to define which examples require which specific solvers and skip those examples when that solver is not available.
P.S. Apologies for the typo in the branch, might cause some confusion when trying to checkout the code and the branch does not seem to exist.