Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent Dependency Handling Across Systems with Different glibc Versions #11120

Open
ido123net opened this issue Jan 30, 2025 · 11 comments
Open
Labels
configuration Settings and such needs-design Needs discussion, investigation, or design

Comments

@ido123net
Copy link

Summary

When using UV to create environments on systems with different glibc versions (e.g., Ubuntu 22.04 and Ubuntu 18.04), packages that depend on glibc (like paramiko) do not handle the version difference gracefully.

In setups where environments are shared across multiple systems via NFS, an environment created on Ubuntu 22.04 functions correctly on the same OS. However, when accessed from Ubuntu 18.04 (which has a lower glibc version), it leads to runtime errors due to incompatible shared library dependencies. The necessary dependencies are not reinstalled or resolved during environment execution.


Steps to reproduce:

  1. On Ubuntu 22.04 (glibc 2.35)
$ uv init example
Initialized project `example` at `/path/to/example`

$ cd example

$ cat hello.py
import paramiko
print(paramiko.__version__)

$ uv add paramiko
Resolved 7 packages in 27ms
Installed 6 packages in 202ms
 + bcrypt==4.2.1
 + cffi==1.17.1
 + cryptography==44.0.0
 + paramiko==3.5.0
 + pycparser==2.22
 + pynacl==1.5.0

$ uv run hello.py 
3.5.0
  1. On Ubuntu 18.04 (glibc 2.27)
$ cd /path/to/example

$ uv run hello.py
Traceback (most recent call last):
  File "/path/to/example/hello.py", line 1, in <module>
    import paramiko
  ...
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /path/to/example/.venv/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so)

Platform

Ubuntu18.04, Ubuntu22.04

Version

uv 0.5.25

Python version

Python 3.9.20

@ido123net ido123net added the bug Something isn't working label Jan 30, 2025
@zanieb
Copy link
Member

zanieb commented Jan 30, 2025

I don't think this is a uv problem, this is just the reality of using packages installed in one context in another context. Paramiko (edit: or, as Charlie subsequently pointed out, a dependency of it) chooses to build wheels that dynamically link libc, so a compatible version of libc needs to be around at runtime.

@zanieb zanieb added question Asking for clarification or support and removed bug Something isn't working labels Jan 30, 2025
@zanieb
Copy link
Member

zanieb commented Jan 30, 2025

(Thanks for the clear reproduction steps though!)

@charliermarsh
Copy link
Member

Hmm... Like, you're accessing the exact same virtual environment? You need different wheels when running against different glibc versions -- that virtual environment isn't seamlessly compatible across platforms. Notice that cryptography publishes wheels for 2.17 and 2.28: https://pypi.org/project/cryptography/#files. uv doesn't introspect which wheel is already-installed when you execute uv run. It could? I doubt any other tool does that.

@charliermarsh
Copy link
Member

@zanieb I think the error is actually in cryptography.

@zanieb
Copy link
Member

zanieb commented Jan 30, 2025

If you install on the older Ubuntu, it should work on the newer one. Alternatively, you could request a specific --python-platform, e.g., one of

x86_64-manylinux2014: An x86_64 target for the manylinux2014 platform. Equivalent to x86_64-manylinux_2_17
x86_64-manylinux_2_17: An x86_64 target for the manylinux_2_17 platform
x86_64-manylinux_2_28: An x86_64 target for the manylinux_2_28 platform
x86_64-manylinux_2_31: An x86_64 target for the manylinux_2_31 platform
x86_64-manylinux_2_32: An x86_64 target for the manylinux_2_32 platform
x86_64-manylinux_2_33: An x86_64 target for the manylinux_2_33 platform
x86_64-manylinux_2_34: An x86_64 target for the manylinux_2_34 platform
x86_64-manylinux_2_35: An x86_64 target for the manylinux_2_35 platform
x86_64-manylinux_2_36: An x86_64 target for the manylinux_2_36 platform
x86_64-manylinux_2_37: An x86_64 target for the manylinux_2_37 platform
x86_64-manylinux_2_38: An x86_64 target for the manylinux_2_38 platform
x86_64-manylinux_2_39: An x86_64 target for the manylinux_2_39 platform
x86_64-manylinux_2_40: An x86_64 target for the manylinux_2_40 platform

where 2_<NUM> refers to the GLIBC version (ref https://peps.python.org/pep-0600/)

But this is only available in the lower-level uv pip interface.

@zanieb
Copy link
Member

zanieb commented Jan 30, 2025

We can think about how to handle this case better though, agree that user-experience is not ideal.

@ido123net
Copy link
Author

Yes, I tried to generalize the use case.

If I'm installing it on the older Ubuntu it works fine on the newer one. (this is actually what we are doing now).
But, I wanted something like the --python-platform for the uv run interface.

The use case is that we have stronger computers with newer OS to develop on, and we have many "old" computers which acts as controllers for some embedded HW.

My CI job is starts on the "strong" PC and distribute the jobs to the "old" PCs.

@charliermarsh
Copy link
Member

Yeah, I think you should be able to specify your minimum-supported glibc version somehow. That would at least solve this case (though wouldn't solve the general case of, like, the platform changing from under us).

@charliermarsh charliermarsh added configuration Settings and such needs-design Needs discussion, investigation, or design and removed question Asking for clarification or support labels Jan 30, 2025
@ido123net
Copy link
Author

I don't know if it's better to let uv-run handle it on its own.

Maybe a better approach would be to handle it like the existing interface in uv-pip, using the --python-platform option.

@charliermarsh
Copy link
Member

Yeah there are basically two options:

  1. Let users specify a minimum glibc version (or --python-platform) as a setting.
  2. In uv run, look at the installed packages, and uninstall + reinstall any packages that are incompatible with the current platform.

(2) would be automatic, but it would also mean that if you're sharing the environment across two incompatible machines in real-time, things would break down.

@ido123net
Copy link
Author

Exactly, this is why I prefer (1), this way I can create the environment with the desire minimum glibc , and share it across all compatible environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
configuration Settings and such needs-design Needs discussion, investigation, or design
Projects
None yet
Development

No branches or pull requests

3 participants