Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add exceptiongroup support to retry_if_exception #501

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

add exceptiongroup support to retry_if_exception #501

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
04ea6f4
add exceptiongroup support to retry_if_exception
jakkdl Oct 31, 2024
7034b62
ruff format
jakkdl Oct 31, 2024
12cc12d
tuple -> typing.Tuple ... though 3.8 is out anyway...
jakkdl Oct 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ jobs:
- python: "3.11"
tox: py311
- python: "3.12"
tox: py312,py312-trio
tox: py312
- python: "3.13"
tox: py313,py313-trio
- python: "3.12"
tox: pep8
- python: "3.11"
Expand Down
1 change: 1 addition & 0 deletions setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ classifier =

[options]
install_requires =
exceptiongroup; python_version < "3.11"
python_requires = >=3.8
packages = find:

Expand Down
7 changes: 7 additions & 0 deletions tenacity/retry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,15 @@

import abc
import re
import sys
import typing

if typing.TYPE_CHECKING:
from tenacity import RetryCallState

if sys.version_info < (3, 11):
from exceptiongroup import BaseExceptionGroup


class retry_base(abc.ABC):
"""Abstract base class for retry strategies."""
Expand Down Expand Up @@ -79,6 +83,9 @@ def __call__(self, retry_state: "RetryCallState") -> bool:
exception = retry_state.outcome.exception()
if exception is None:
raise RuntimeError("outcome failed but the exception is None")
if isinstance(exception, BaseExceptionGroup):
# look for any exceptions not matching the predicate
return exception.split(self.predicate)[1] is None
Copy link

@LotfiRafik LotfiRafik Feb 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that retry will not happen if the exception group contains both the exception for which we should retry and other exceptions.. :(
i prefer to retry if the exception group contains one of the configured exception(s) :

return exception.subgroup(self.predicate) is not None

or make it parametrable ("contains", "only"..)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm. I can see arguments either way

I think current implementation is good because it requires you to be explicit about all the errors you expect, and if you suddenly start seeing new errors you'll be notified

but if one specifically wants "retry if you get TimeOutError, but I don't care if I get X, Y, Z" and you specify retry_if_exception_type((TimeOutError, X, Y, Z)) then you're also retrying if you only get X/Y/Z.

The latter does bring additional pitfalls of swallowing e.g. KeyboardInterrupt, though there's plenty stuff in this library that will already do that.

though there's always the option of just manually doing

@retry(
    retry=retry_if_exception(
        lambda e: isinstance(e, ExceptionGroup) and e.subgroup(ValueError) is not None
    )
)
def test_foo():
    ...

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree (specially with the possibility of always having the choice to set manually the retry predicate)

return self.predicate(exception)
else:
return False
Expand Down
44 changes: 44 additions & 0 deletions tests/test_tenacity.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@
import tenacity
from tenacity import RetryCallState, RetryError, Retrying, retry

if sys.version_info < (3, 11):
from exceptiongroup import ExceptionGroup

_unset = object()


Expand Down Expand Up @@ -733,6 +736,24 @@ def go(self):
return True


class NoExceptionGroupAfterCount:
def __init__(self, count: int, exceptions: typing.Tuple[Exception]):
self.counter = 0
self.count = count
self.exceptions = exceptions

def go(self):
"""Raise an ExceptionGroup until after count threshold has been crossed.

Then return True.
"""
if self.counter < self.count:
self.counter += 1
raise ExceptionGroup("tenacity test group", self.exceptions)

return True


class NoNameErrorAfterCount:
"""Holds counter state for invoking a method several times in a row."""

Expand Down Expand Up @@ -1014,6 +1035,7 @@ def _retryable_test_with_exception_type_custom(thing):
retry=tenacity.retry_if_exception_type(CustomError),
)
def _retryable_test_with_exception_type_custom_attempt_limit(thing):
# this is not used??
return thing.go()


Expand Down Expand Up @@ -1064,6 +1086,28 @@ def test_retry_if_exception_of_type(self):
self.assertTrue(isinstance(n, NameError))
print(n)

def test_retry_if_exception_of_type_exceptiongroup(self):
self.assertTrue(
_retryable_test_with_exception_type_io(
NoExceptionGroupAfterCount(5, exceptions=(IOError(),))
)
)
with pytest.raises(ExceptionGroup):
self.assertTrue(
_retryable_test_with_exception_type_io(
NoExceptionGroupAfterCount(5, exceptions=(IOError(), ValueError()))
)
)
# not supported
with pytest.raises(ExceptionGroup):
e = IOError()
e.__cause__ = NameError()
self.assertTrue(
_retryable_test_with_exception_cause_type(
NoExceptionGroupAfterCount(5, exceptions=(e,))
)
)

def test_retry_except_exception_of_type(self):
self.assertTrue(
_retryable_test_if_not_exception_type_io(NoNameErrorAfterCount(5))
Expand Down
2 changes: 1 addition & 1 deletion tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tox]
envlist = py3{8,9,10,11,12,12-trio}, pep8, pypy3
envlist = py3{8,9,10,11,12,13,13-trio}, pep8, pypy3
skip_missing_interpreters = True

[testenv]
Expand Down