Installing from file:// on Windows with optional dependencies via @ doesn't work

[larsoner]

Description

I want to install a .whl file locally with an optional dependency on Windows, but it does not seem possible. Works fine on macOS and Linux.

Expected behavior

Install mne-lsl from a local wheel file with [test] optional deps.

pip version

24.2

Python version

3.12.4

OS

Windows 10

How to Reproduce

Get any .whl file locally, but the one I'm using in particular is from this failing Windows GitHub action CI, available on the Summary page.

This command works as expected:

$ pip install --dry-run "file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"
Processing z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl
Requirement already satisfied: click>=8.1 in c:\users\tester\mne-python\1.7.1_00\lib\site-packages (from mne-lsl==1.7.0.dev0) (8.1.7)
...
Requirement already satisfied: six>=1.5 in c:\users\tester\mne-python\1.7.1_00\lib\site-packages (from python-dateutil>=2.7->matplotlib>=3.5.0->mne>=1.4.2->mne-lsl==1.7.0.dev0) (1.16.0)
Would install mne_lsl-1.7.0.dev0

and so does this (output suppressed for brevity):

$ pip install --dry-run "Z:\\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"

However, prefixing mne_lsl[test] @ does not:

$ pip install --dry-run "mne_lsl[test] @ file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"
Processing z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl (from mne_lsl[test]@ file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl)
ERROR: mne_lsl@ file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl from file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl (from mne_lsl[test]@ file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl) does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

and neither does this, but fails in a different way:

$ pip install --dry-run "mne_lsl[test] @ Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\base_command.py", line 105, in _run_wrapper
    status = _inner_run()
             ^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\req_command.py", line 67, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\commands\install.py", line 379, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 539, in collect_root_requirements
    reqs = list(
           ^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 495, in _make_requirements_from_install_req
    cand = self._make_base_candidate_from_link(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 232, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 281, in __init__
    assert name == wheel_name, f"{name!r} != {wheel_name!r} for wheel"
           ^^^^^^^^^^^^^^^^^^
AssertionError: 'mne-lsl' != '\\mne-lsl' for wheel

So I don't see a way to install a wheel file locally on Windows with optional dependencies. Even if there is another way, it seems like at least the file:// version above should work.

This one works but does not guarantee that it will install the wheel from the desired file (if it doesn't find one it likes, it will pull from PyPI rather than just failing; and --no-links will make it so that deps are not resolved):

$ pip install --dry-run --find-links "Z:\\" --pre "mne_lsl[test]"

Output

☝️

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

I've not tried this locally yet, but from my past experience of installing local packages with extras, I'm wondering if this works:

pip install --dry-run "file://Z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"[test]

? (or some variation of putting the quotes in the right place)

[larsoner]

Yes that seemed to work locally!

[larsoner]

... well I should clarify, this works:

$ pip install --dry-run "Z:\\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"[test]
Processing z:\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl (from mne-lsl==1.7.0.dev0)
Requirement already satisfied: click>=8.1 in c:\users\tester\mne-python\1.7.1_00\lib\site-packages (from mne-lsl==1.7.0.dev0->mne-lsl==1.7.0.dev0) (8.1.7)
...
Requirement already satisfied: six>=1.5 in c:\users\tester\mne-python\1.7.1_00\lib\site-packages (from python-dateutil>=2.7->matplotlib>=3.5.0->mne>=1.4.2->mne-lsl==1.7.0.dev0->mne-lsl==1.7.0.dev0) (1.16.0)
Using cached pytest_randomly-3.16.0-py3-none-any.whl (8.4 kB)
Would install mne_lsl-1.7.0.dev0 pytest-randomly-3.16.0

The file:// does not, in yet another different way!

$ pip install --dry-run "file://Z:\\mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl"[test]
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\base_command.py", line 105, in _run_wrapper
    status = _inner_run()
             ^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\req_command.py", line 67, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\commands\install.py", line 343, in run
    reqs = self.get_requirements(args, options, finder, session)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\cli\req_command.py", line 233, in get_requirements
    req_to_add = install_req_from_line(
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\req\constructors.py", line 405, in install_req_from_line
    parts = parse_req_from_line(name, line_source)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\req\constructors.py", line 308, in parse_req_from_line
    if is_url(name):
       ^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\site-packages\pip\_internal\vcs\versioncontrol.py", line 54, in is_url
    scheme = urllib.parse.urlsplit(name).scheme
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\urllib\parse.py", line 500, in urlsplit
    _check_bracketed_host(bracketed_host)
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\urllib\parse.py", line 446, in _check_bracketed_host
    ip = ipaddress.ip_address(hostname) # Throws Value Error if not IPv6 or IPv4
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tester\mne-python\1.7.1_00\Lib\ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: 'test' does not appear to be an IPv4 or IPv6 address

[notatallshaw]

Interesting, there are definetly known limitations to pip's ability to parse local requirements, and I think both Windows and @ are problamatic.

This might be a duplicate of an existing issue, I'll have a search around later if no one else knows else chimes in.

[barneygale]

I think the URL should be file:///Z:/mne_lsl-1.7.0.dev0-cp310-abi3-win_amd64.whl. Note the extra slash at the beginning, and the use of a forward slash after Z:

[larsoner]

Ahh, so the choice of file:///Z:/... is intentional because it's a URL-reformatting of the Windows path Z:\...? A bit of a bummer since tools like cygpath won't just automatically translate for me but it's workable. Should I open a PR to document somewhere how a Windows path should be formatted? Maybe updating point 10 here which doesn't seem to be correct in the Windows tab I think?

https://pip.pypa.io/en/stable/cli/pip_install/#examples

[pfmoore]

From what I recall, there's so many conflicting documents and implementations on how to express arbitrary paths as URLs (not only Windows paths with drive names, but relative paths and network shares are hard to find good documentation for) that I'd be reluctant to try to document it here. Maybe we could refer the user to the stdlib docs for Path.as_uri(), which aren't very explicit, but at least give a way to find a valid URI for a given path?

[barneygale]

https://datatracker.ietf.org/doc/html/rfc8089 is a decent reference, though it's mostly appendices. Here's a table showing how many slashes should be added to the beginning of a path when forming a file: URI.

	No change	Add 1 slash	Add 2 slashes	Add 3 slashes
foo/bar	Preferred
/foo/bar	OK		Preferred
//foo/bar (POSIX)			Preferred
//foo/bar (Windows)	Preferred		OK	OK
C:/foo/bar (Windows)	OK	OK		Preferred

The cells labelled "OK" are acceptable but usually considered variations.