Union should work for constrained type variable

[hannes-ucsc]

https://mypy-play.net/?mypy=master&python=3.12&gist=f0db7bce0a6d929cff485bd50c76a158

def truncate[S:(str, bytes)](s: S) -> S:
    return s[:10]
    
def log(v: str | bytes):
    print(truncate(v))  # error: Value of type variable "S" of "truncate" cannot be "str | bytes"  [type-var]

How can I avoid the error without narrowing to every possible alternative of the union (note how the if and else branches are identical) …

def log2(v: str | bytes):
    if isinstance(v, str):
        print(truncate(v))
    else:
        print(truncate(v))

… or adding overloads to truncate? Both seem like inelegant and tedious workarounds.

[Daverball]

If you want to be able to pass in an union you should use an upper bound of str | bytes, rather than a constrained type variable.

It's rare that you actually need a constrained type variable. They're not very well specced out currently, so you can expect many bugs and weird corner cases.

def truncate[S: str | bytes](s: S) -> S:
    return s[:10]
    
def log(v: str | bytes):
    print(truncate(v))

I see that this version leads to a different error in mypy for return s[:10], probably because bytes.__getitem__ is overloaded. This error is a bug however, so you can safely type ignore it and open an issue in the mypy bug tracker, if there isn't already one.

[hannes-ucsc]

Thank you! If it's a bug in mypy, it's not because of the overload:

https://mypy-play.net/?mypy=master&python=3.12&gist=89357de9ce83741a67351b0ed4f1e6ff

class A:
    def f(self) -> A:
        return A()

class B:
    def f(self) -> B:
        return B()

def truncate[S: A | B](s: S) -> S:
    return s.f()  # error: Incompatible return value type (got "A | B", expected "S")  [return-value]
    
def truncate2[S: (A, B)](s: S) -> S:
    return s.f()

I am stupefied. Do you still recommend filing a bug against mypy? I hesitate because clearly I am not getting something about the difference between the bound S: A | B and the constraint S: (A, B). To me they are equivalent, but if they are, why do we even have constraints?

[hauntsaninja]

One difference is in how subtypes (like subclasses) and supertypes (like unions) are handled.

def truncate[S: A | B](s: S) -> S:
    return s.f()  # error: Incompatible return value type (got "A | B", expected "S")  [return-value]

With a (bounded) TypeVar like in the above, truncate promises that the return value will be the same type as the input value (assuming it meets the bound).

However, if you have class C(B): ... then truncate(C()) will not return a C (it'll return a B) and hence you get the complaint.

"constrained" or "value restricted" type variables will only solve to one of the types they are constrained to. i.e. instead of solving S=C or S=A | B, type checkers will only attempt to solve S=A or S=B.

Constrained type variables were added to the type system to model some Python semantics involving operators and subclasses. This specifically mattered for modeling functions involving str and bytes, especially in stubs, which especially mattered for folks using type checking as a means of migrating from Python 2 to 3. See also https://mypy.readthedocs.io/en/stable/generics.html#type-variables-with-value-restriction

[hannes-ucsc]

With a (bounded) TypeVar like in the above, truncate promises that the return value will be the same type as the input value (assuming it meets the bound).

Do you mean the promise is type(truncate(s)) is type(s) or is it isinstance(truncate(s), type(s))?

[Daverball]

Right, I hadn't considered the fact that str and bytes don't return Self for most methods but rather the base class. This includes slicing, so:

class MyStr(str):
    pass

reveal_type(MyStr("")[:])  #  str, not MyStr

So the value constrained version is actually more correct for str and bytes, since it can't specialize the generic to a subclass:

class MyStr(str):
    pass

def truncate_bounded[S: str | bytes](s: S) -> S:
    return s[:10]

def truncate_constrained[S: (str, bytes)](s: S) -> S:
    return s[:10]
    
    
reveal_type(truncate_bounded(MyStr("")))  # should be `str`, but actually is `MyStr`
reveal_type(truncate_constrained(MyStr("")))  # is `str`, as it should be, in order to be safe

So mypy is actually correct in both cases, because truncate can't be specialized to str | bytes, you're not allowed to pass in str | bytes, since that is a supertype of both str and bytes. You could add a third value constraint for str | bytes to get around this or add a runtime check in order to ensure that truncate isn't buggy:

def truncate[S: str | bytes](s: S) -> S:
    result = s[:10]
    assert isinstance(result, type(s))  # ensures we don't return something other than Self
    return result

[hauntsaninja]

The other trick not yet mentioned is just use overload. It's a little more verbose, but you'll get good results, and it's quite straightforward to reason about