pip 22.0.1 broke `pip_shims.unpack_url`

[KerberosMorphy]

Steps to reproduce

Hard to give a way to reproduce since I make lots of experimentation in Docker at the same time.
I've tried do recreate a small example but I couldn't.

In all case, the code speak for itself.

Current situation

Error raised since pip expect verbosity as fourth positional argument.

[TypeError]: unpack_url() missing 1 required positional argument: 'verbosity'

pdm/pdm/models/candidates.py

Lines 257 to 262 in afb3244

	downloaded = pip_shims.unpack_url(
	self.link,
	ireq.source_dir,
	downloader,
	hashes=ireq.hashes(False),
	)

Change in pip code

They change the behavior of unpack_url from 21.3.1 to 22.0.1:

def unpack_url(
    link: Link,
    location: str,
    download: Downloader,
+   verbosity: int,
    download_dir: Optional[str] = None,
    hashes: Optional[Hashes] = None,
) -> Optional[File]:

https://github.com/pypa/pip/blob/c73ac8d6bcf4f64041cafeccd2125cca052abed9/src/pip/_internal/operations/prepare.py#L174-L181

Environment Information

PDM version:        1.12.6
Python Interpreter: /usr/local/bin/python (3.9)
Project Root:       /app
Project Packages:   /app/__pypackages__/3.9
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.10",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.10.60.1-microsoft-standard-WSL2",
  "platform_system": "Linux",
  "platform_version": "#1 SMP Wed Aug 25 23:20:18 UTC 2021",
  "python_full_version": "3.9.10",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

[H-Plus-Time]

In the meantime, a workaround is to specify the pip dependency in your install-pdm step:
curl -sSL https://raw.githubusercontent.com/pdm-project/pdm/main/install-pdm.py | python3 - --dep "pip<22.0.1"

That or the other ways to specify that dependency override (the PDM_DEPS env var for instance).

[CharString]

Maybe we should consider pinning the pdm dependencies. That would prevent these breakages, at least when installed with pipx.

PS to fix a broken pipx installation

$ pipx runpip pdm install --upgrade 'pip<22.0'

[pawamoy]

Maybe we should consider pinning the pdm dependencies. That would prevent these breakages

The choice was made not to pin them. See https://iscinumpy.dev/post/bound-version-constraints/.
Pinning them would indeed prevent such breakages, but these breakages are fixable downstream: you showed a workaround 🙂
If we pin, bugfixes and security fixes on other dependencies are made inaccessible! There's no downstrem workaround! Users have to wait until maintainers update their pins and push a release.

After reading and thinking about the article mentioned above, I see the act of pinning dependencies as "securing the past", while unpinning (and removing upper bounds) as "easing the future". And I prefer the latter. Just my opinion!

[CharString]

@pawamoy I agree with the article, that you shouldn't always pin upper bound versions. For the exceptions frequent pdm update in the right context helps keep my dependencies flowing and my users and colleague from getting confused.

I don't like how pipenv vendors half of pypi. But it does recognize that it's a tool, an application, a command that people count one to be working. I think wisdom lies somewhere in between on this spectrum.

PS in the time we spent bickering over this, either of us could have written a PR that solves it. :)

[pawamoy]

@CharString I don't think it was bickering: you suggested considering pins on pip, I gave my opinion (and the reason why it's currently not upper-bounded) 😄 I believe these sort of exchanges are valuable 🙂
Also, you're right, in some cases upper-bounds (or even pins) can be useful, I kinda gave the extreme view of "never pin/bound".
I'll let @frostming judge if pip should be bound.
And thanks @q0w for the PR!

[pawamoy]

Just for reference: pypa/pip#9639

[frostming]

I think pinning pip<22.0 is acceptable for fixing this and it is a chance to accelerate the process of dropping pip #743 .