Is it possible to share the platform site pckages while using a local virtual environment?

[huxuan]

When working in dev containers, it is likely to share some fundamental dependencies. For example, when using the NGC PyTorch Containers, we prefer to share the pytorch and some other packages. In the meanwhile, we still want to keep local virtual environment especially when working on different projects.

I investigated a little bit about the global project related features, but it does not seems to meet the requirements. Just let me know if I missed anything. I also found there is a --site-packages option for pdm run, but we also need that for install and other commands.

This may bring extra challenages when locking dependencies, I am ok if there is any workaround for it.

BTW, pipenv has a similar feature with --site-packages: https://pipenv.pypa.io/en/latest/advanced.html#working-with-platform-provided-python-components

[huxuan]

It seems that there is a config "include-system-site-packages" [1] for venv, but there is no related documentation and it is also not shown in the pdm config.

[1]

pdm/src/pdm/models/venv.py

Line 74 in f429652

def include_system_site_packages(self) -> bool:

[frostming]

That's a config of venv creation. In fact you can give abitrary venv arguments to pdm venv create command:

pdm venv create -- --system-site-packages

[huxuan]

Hi @frostming, thanks for the response.

Accroding to the hints you provide, I find #2287 that is really helpful. pdm venv create python --system-site-packages with --update-reuse-installed works for me. Seems the positional argument python is needed to refer to the default python.

Just some non-critical inconvenienve I found:

• pdm list does not differentiate the system and local site packages, the location field seems to be the place, but all of them are empty.
• If I create the venv --with-pip, pdm run pip list will not show the packages installed by pdm.

Let me know if this is by design or anything I can do to help. Thanks for creating the great tool!

[frostming]

the location field seems to be the place

This field has value only if the dependency is installed from a local directory.

[frostming]

• If I create the venv --with-pip, pdm run pip list will not show the packages installed by pdm.

Not true:

❯ pdm run pip list
Package      Version Editable project location
------------ ------- ----------------------------------------
blinker      1.8.2
click        8.1.7
Flask        3.0.3
itsdangerous 2.2.0
Jinja2       3.1.4
MarkupSafe   2.1.5
myproj       1.1.1   /Users/fming/wkspace/playground/test-pdm
pip          24.0
Werkzeug     3.0.3

[huxuan]

This field has value only if the dependency is installed from a local directory.

OK, if it is by design, the differentiation would be a feature request then. :-)

Not true

Sorry, it seems that I have messed up with the system-level pip. I can confirm it works.

Thanks!