Python Packaging 2023

[pawamoy]

Hey @frostming, I've read your "New developments in Python packaging" article (translated to english), and I wanted to answer here to the questions you ask the readers.

First, it's delightful to hear that PEP582 has progressed (lets hope in the right direction). Being able to install with pip install --prefix __pypackages__ would be very nice (I've been using pip install -t __pypackages__/3.X/lib before, with the limitations it has).

I'll add a comment for each point I want to comment on, to enable a threaded discussion 🙂

[pawamoy]

I also plan to change the path to the latest standard in the next major update, let me know in advance.

Totally fine with that change. I don't use Windows and so don't really care about it, but obviously (to me at least) it would be better if Windows' paths had the version number in them as well.

[pawamoy]

If the directory where the script is located is not found __pypackages__, do you want to find it in the parent directory and ancestor directory?

I have nothing against that (seems like a common thing, as you point in the article), but I generally don't use that feature either. I very, very rarely enter subdirectories in my own projects, and run everything from the root of the repository, using make or other task runners.

[pawamoy]

Do you want to disable system site-packages when __pypackages__ is loaded?

YES. For once, I have a strong opinion on this. At least, IMO, it should be disabled by default, and an option could tell PDM to enable it. My reasoning is: isolation above all. If I want to use the fictive bpython, or existing tools like ipython from within my project and its dependencies, then I list it as a dev-dependency, period. Python (virtual) environments have been confusing enough, I don't want to ever think again about paths/libs leaking from one environment to the other. If I want to install a tool globally, for example with pipx, then I accept that I can only run "static" tasks in my projects (code analysis, etc.) without runtime access to the projects dependencies. The only tool that makes an exception to this rule is PDM itself.

EDIT: I've resolved the following issue, don't pay attention to it.

---

To back this up, I recently had a new contributor on a project, and he got errors running my linting task because he had flake8 installed globally with additional plugins, and they were leaking into the project. Very bad developer experience IMO 😕

What I don't understand is that the help message of pdm run seems to indicate that the interpreter site-packages are only loaded when the -s option is used. But apparently they're always loaded?

UPDATE: found the explanation.

• pdm run flake8 --bug-report will indeed not load the interpreter site-packages
• but python -m flake8 --bug-report (PEP582 enabled globally) will load them

It means one must be very careful how they run commands/tasks, because depending on how you invoke a module/script, the interpreter site-packages could leak into the current environment. I'll have to run some tests with my task runner to find a way and make sure that never happens again 🙂