Add --excludes to run_benchmarks.rb to easily exclude benchmarks #448
+38
−15
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
--categoryand--name_filtersdon't give a way to exclude some benchmarks.--name_filtersseems already quite complicated so adding some negated filter there seems undesirable.Therefore I think it's best to add an explicit
--excludes=x,y,zoption.Thanks to the recent refactor, this is pretty easy :)
There are many possible reasons to exclude a given benchmark notably too unstable performance, known to fail on some Ruby version and no point to even
bundle install, etc.This is vastly more convenient than having to manually exclude by including every other benchmark by name, especially as more benchmarks are added.