good way to record which specs have been re-tried?

[tansaku]

I was hoping to use rspec-retry to not only automatically re-run failing tests, but to record information about which tests are failing intermittently and which are failing multiple times. Does that information get summarised anywhere?

I was thinking of using the retry_callback but that only runs in between tests, and so means it can only be used to record individual failures, and not used to indicate that a test first failed and then passed. Is there some other mechanism available?

Many thanks in advance

[shotop]

Found myself here wondering about a similar strategy. Ideally the retry is never used, but when it is, it would be awesome to have some way to know which specs were retried without having to traverse through the build output. With this info, I could target the specs that were flakey. Thanks in advance for any advice and thanks for an awesome tool.

[smccarthy]

I am in a similar situation. I was using examples = RSpec.world.filtered_examples.values.flatten in my after suite hook. However, this doesn't include all the tries of a test (only the last one I think).

[rymai]

For reference, we at GitLab developed such functionality: https://gitlab.com/gitlab-org/gitlab/-/blob/master/tooling/rspec_flaky/listener.rb

Maybe we could upstream it to this gem?

[megatux]

There is a PR for adding a callback to do something on failure.
Our company maintain a fork with it here