Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip compile ---no-emit-package cannot produce consistent results #11058

Open
ssbarnea opened this issue Jan 29, 2025 · 2 comments
Open

pip compile ---no-emit-package cannot produce consistent results #11058

ssbarnea opened this issue Jan 29, 2025 · 2 comments
Labels
bug Something isn't working needs-mre Needs more information for reproduction

Comments

@ssbarnea
Copy link

ssbarnea commented Jan 29, 2025

Summary

As ruamel-yaml library has an optional binary dependency named "ruamel-yaml-clib" that is not already buildable for all platforms (some arm64 are always missing), we discovered that using `uv pip compile --universal" does not produce consistent results and they vary based on the architecture/platform on which you run the command.

In the past with pip-tools's pip-compile we successfully excluded this dependency when building, but apparently the no-emit-package feature in uv does work in different ways and the resulted constraints file varies depending on where you run it:

# The following packages were excluded from the output:
# ruamel-yaml-clib
[tool.uv.pip]
annotation-style = "line"
no-emit-package = [
  "ruamel-yaml-clib",
]

I would not mind if this is included or not but I want to ensure that when we run the compile command we get the same output file.

What makes it even worse is that there is no option to disable this annotation. We have styles and options for no header, but no option to disable this trailing comment.

Fixing any of these could allow us to produce consistent constraint files.

Please note that this is not the only case where the resulting exclude list varies from machine to machine. I already found 3 other cases. The trick is to find a way to either disable producing it, or always include all the packages that were mentioned in the configuration (regardless if compile command really had to remove them or not), as any of this will ensure consistent results.

Platform

macos

Version

0.5.23

Python version

3.13.1

@ssbarnea ssbarnea added the bug Something isn't working label Jan 29, 2025
@ssbarnea
Copy link
Author

Looking at the source code from

if !excluded.is_empty() {
i realize that as long I want to exclude packages from the produced constraints file, there is no way to avoid these trailing comments.

@ssbarnea ssbarnea changed the title pip compile cannot produce reproducible results on different platforms due to excludes pip compile ---no-emit-package cannot produce consistent results Jan 29, 2025
@charliermarsh
Copy link
Member

Are you able to provide a specific minimal example I can use to reproduce? Looking at the code, it seems "impossible" to me that the results would vary by host machine if you're using --universal.

(It does seem like the annotation is only present if the package is actually excluded, i.e., it otherwise would've been part of the resolution. But that behavior seems right to me?)

@zanieb zanieb added the needs-mre Needs more information for reproduction label Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs-mre Needs more information for reproduction
Projects
None yet
Development

No branches or pull requests

3 participants