Exact return types instead of shutil._PathReturn

[Avasam]

Closes #13508

#13508 (comment)

Given that the return type would be StrPath (ie: str | PathLike[str]) and that we can return a str if dst: str and str | _T otherwise, I'd argue that's a fine union to return.

Most cases of using a StrPath whether you actually have a str or a PathLike won't matter too much. If you do explicitly need a str after potentially passing a PathLike or explicitely need the same PathLike you passed, then you should really check the return here.

And once again, in most cases, you can coerce the type (using str() or Path())

>>> Path("")
WindowsPath('.')
>>> Path(Path())
WindowsPath('.')
>>> str(".")
'.'
>>> str(Path("."))
'.'

(this argument may not hold as well for bytespath though, what's a common PathLike[bytes] ?)

---

I'm not hard set on this, it's a balance between type safety, and a (imo) possibly very small annoyance in the specific case you know for certain that you're moving/copying a file + are passing in a PathLike + need the result to not be a string.

But I'm at least demonstrating the suggestion, and can dial it back to an Any return type.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/filesystems.py:162: error: Value of type variable "_StrPathT" of "copytree" cannot be "Optional[str]"  [type-var]
- src/prefect/filesystems.py:162: error: Argument 2 to "copytree" has incompatible type "Optional[str]"; expected "Union[str, PathLike[str]]"  [arg-type]

mkosi (https://github.com/systemd/mkosi)
+ mkosi/__init__.py:3307:16: error: Incompatible return value type (got "tuple[Union[Path, str], None, None, list[Path]]", expected "tuple[Optional[Path], Optional[str], Optional[Path], list[Path]]")  [return-value]
+ mkosi/__init__.py:3320:12: error: Incompatible types in assignment (expression has type "Union[Path, str]", variable has type "Path")  [assignment]
+ mkosi/__init__.py:3329:12: error: Incompatible types in assignment (expression has type "Union[Path, str]", variable has type "Path")  [assignment]

[srittau]

Obligatory link to python/typing#566 (AnyOf suggestion).

[Avasam]

Obligatory link to python/typing#566 (AnyOf suggestion).

Would love to see an official proposal for that one. I don't think I have the time/motivation/expertise to tackle that though.

[srittau]

Considering the low primer impact and the fact that this is a genuine trap (and a bug in my opinion) this change seems good to me.

[behrmann]

Just for reference: This broke us downstream in mkosi. This was fixed with systemd/mkosi#3675. For us the change was rather small, but other people might be broken more significantly (or maybe not looking at your mypy_primer bot) and some more dependant typing that keeps the type of what is passed through would be nice—though I don't know if shutil guarantees this.

Regarding your mypy_primer action: If you see us affected by a change, feel free to reach out, we have a largeish stdlib-only fully-typed codebase that's running current mypy and pyright pretty much daily. We are a bit limited by our current minimum required Python (3.9) for the moment, but we strive to be as fully typed with as few casts, coercions or ignores as possible.

[srittau]

@behrmann I'm sorry for the breakage. I'll try to keep in mind to contact mkosi when we notice primer breakage. Unfortunately, this area of shutil is a bit inconsistent regarding pathlib support. That pathlib works at all is probably incidental as the code predates the introduction of pathlib. This leads to code like this, where under some (external) circumstances str is returned, and under some other circumstances, the passed in Path object is returned, which makes it hard to type:

https://github.com/python/cpython/blob/67ded6a4faae29edff8e4f7886978e71ce116e33/Lib/shutil.py#L468-L484

This should probably fixed in CPython. I've opened python/cpython#132322 for now to track this at least.