[Wheel Variants] Wheel Variant Support for PIP #13366
Conversation
Add parameters to include variant hashes in supported tag list only if any wheels found have variant hashes. Fix supported tag list caching to keep separate caches per parameters. Start preparing for `variants.json` support. The design is aimed at supporting three scenarios: 1. Package version has no variant hashes -- we do not fetch `variants.json` and want tags without variants (much faster). 2. Package version has variant hashes -- we try to fetch `variants.json`, if we have it, then we want tags with these variants (faster). 3. Package version has variant hashes but no `variants.json` -- fallback to full list of tags with all possible variants (slow). Currently only 1. and 3. are implemented.
Add initial support for fetching and passing `variants.json`. The file is currently fetched (and cached), and passed to construct tags. However, variant hashes are neither filtered nor sorted yet. For the time being, I had to remove `@cache`, since it's incompatible with dict parameters.
[WheelVariant] `variants.json` support prototype
…json Merge variants-json sub-branch into the main branch
Remove the support for combinatorial approach to variants, leaving only `variants.json` support. Remove the redundant `need_variants` parameter, as the use of variants is now governed by the presence of `variants_json`.
DEKHTIARJonathan
force-pushed
the
pep-xxx-wheel-variants
branch
from
54c7ebc to
3292f9a
Compare
|
What's the motivation for making this a PR to pip, as opposed to having it in a branch in your own fork of pip? It's nowhere near ready for inclusion (apart from anything else, it's blocked behind a standards discussion that's likely to be pretty extensive) and the pip maintainers are unlikely to do anything with it. As far as I can tell, at this point in the feature's development, having a PR here has no real value, and it adds to the backlog of outstanding PRs for pip, which is already far too long 🙁 |
|
@warsaw suggested it was a good thing for people to be able to track development. If it's not helpful I'm happy to close it and reopen it when ready |
|
Personally, I think it's missing too much context - most of the wheel-next work is going on in dedicated channels, which means that most people aren't going to know any of the background about what all this means (I know that's true in my case). That's just my perspective, though - let's see what the other @pypa/pip-committers think before closing this. |
|
For context we are still very much in the development / design phase of the proposal. We are approaching a stable design that we feel confident enough to draft as a PEP. We expect to have the PEP ready around summer. If anyone is interested to know more about it or even to just try without getting into the implementation details - we have tutorials published under https://github.com/wheelnext/pep_xxx_wheel_variants/blob/main/README.md There are a lot of moving pieces in this proposal (from build backends to package indexes passing by installers). Coming up with a complete proposal that has been tested by many people, many projects, many installers, different perspectives just take time. In turn that should produce a high quality (we hope) PEP and help the review process and discussion. WheelNext channels (https://discord.com/channels/803025117553754132/1279204588196597811), meetings, emails (https://mail.python.org/mailman3/lists/wheelnext-announce.python.org/) are fully opened and we encourage anyone interested to participate. We just don't want to spam everybody with long design discussions and multiple design iterations. All the slides are available here: https://wheelnext.dev/slidedecks_resources/ We are trying to be respectful of everybody's time. If this PR is not helping - happy to close |
DISCLAIMER: This is a work in progress - This development is being conducted as part of the WheelNext initiative.
https://github.com/wheelnext
This PR is for now only for illustration to highlight the full implementation details of Wheel Variants inside PIP.
I don't expect this PR to be fully functional without
variantlib: https://github.com/wheelnext/variantlib/ vendored inside PIP.PR Authors: @mgorny @DEKHTIARJonathan
CC: @rgommers @warsaw @emmatyping @aterrel @msarahan