Encountering exception when using internal artifactory pypi repo

[racc]

Hi all,

I'm encountering a potential issue when trying to connect to a company internal artifactory pypi repo which requires authentication.

The exception encountered is:
pdm.exceptions.PdmException: The credentials for artifactory.example.com are not provided. Please rerun the command with -v option.

Contents of ~/.config/pdm/config.toml

[pypi]
url = "https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple"
verify_ssl = false

[pypi.extra]
url = "https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-extra/simple"
verify_ssl = false

PDM version:
2.4.7
Python Interpreter:
/opt/rh/rh-python38/root/usr/bin/python (3.8)

When I run pdm install -v and enter credentials and save them to the keyring the build succeeds, however I would like to run this in non-interactive mode and would expect that PDM should be able to work in this way.

I have also tried adding username and password fields to the config.toml file but this results in:
401 Error Credentials not correct.

Is this a potential bug, or am I doing something wrong here?

Thanks in advance,
Jason

[frostming]

Try a different config pattern:

[pypi]
url = "https://artifactory.example.com/artifactory/api/pypi/pypi-remote/simple"
verify_ssl = false
username = "<username>"
password = "<password>"

[racc]

I did try this, however specifying the username and password fields fails to authenticate, and returns this error:
401 Error Credentials not correct

For some reason, it only works when the credentials are embedded in the URL?

[frostming]

Note that when they are embedded in the URL, they must be percent-encoded. For example, if the password is 123%456, it will appear as 123%25456, while in the password field, it is as-is. Just a kind reminder if you are not aware.

[racc]

Thanks for the info. The password is generated by Artifactory and doesn't require URL encoding - it only uses [A-Za-z0-9] characters.
https://www.jfrog.com/confluence/display/JFROG/PyPI+Repositories#PyPIRepositories-ResolvingfromArtifactoryUsingpip

[racc]

FYI the redacted PDM log looks like this:

pdm.termui: Installing setuptools 67.5.1
unearth.auth: Found index url https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple
unearth.auth: Found index url https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple
unearth.auth: Getting credentials from keyring for url: https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple
unearth.auth: Getting password from keyring for: __token__@https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple
unearth.auth: Getting credentials from keyring for url: artifactory.example.com
unearth.auth: Getting password from keyring for: [email protected]
pdm.termui: Error occurs
Traceback (most recent call last):
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/termui.py", line 236, in logging
    yield logger
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 405, in synchronize
    self.install_candidate(self_key, progress)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 230, in install_candidate
    self.manager.install(can)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/installers/manager.py", line 33, in install
    installer(str(prepared.build()), self.environment, prepared.direct_url())
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/models/candidates.py", line 375, in build
    self.wheel = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/builders/editable.py", line 45, in build
    self.install(self._requires, shared=True)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/builders/base.py", line 294, in install
    install_requirements(missing, env)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/installers/core.py", line 45, in install_requirements
    manager.install(candidate)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/installers/manager.py", line 33, in install
    installer(str(prepared.build()), self.environment, prepared.direct_url())
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/models/candidates.py", line 360, in build
    self.obtain(allow_all=False)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/models/candidates.py", line 419, in obtain
    result = finder.download_and_unpack(self.link, build_dir, download_dir, hash_options)
  File "/home/<username>/.local/lib/python3.8/site-packages/unearth/finder.py", line 359, in download_and_unpack
    file = unpack_link(
  File "/home/<username>/.local/lib/python3.8/site-packages/unearth/preparer.py", line 300, in unpack_link
    resp = session.get(link.normalized, stream=True)
  File "/home/<username>/<proj path>/__pypackages__/3.8/lib/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/home/<username>/<proj path>/__pypackages__/3.8/lib/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/<username>/<proj path>/__pypackages__/3.8/lib/requests/sessions.py", line 708, in send
    r = dispatch_hook("response", hooks, r, **kwargs)
  File "/home/<username>/<proj path>/__pypackages__/3.8/lib/requests/hooks.py", line 30, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/home/<username>/.local/lib/python3.8/site-packages/unearth/auth.py", line 309, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
  File "/home/<username>/.local/lib/python3.8/site-packages/pdm/models/auth.py", line 46, in _prompt_for_password
    raise PdmException(
pdm.exceptions.PdmException: The credentials for artifactory.example.com are not provided. Please rerun the command with `-v` option.

Perhaps the issue lies with unearth?

[frostming]

These two lines indicate URL matches the index URL, but the auth isn't extracted or not valid

unearth.auth: Found index url https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple
unearth.auth: Found index url https://<username>:<password>@artifactory.example.com/artifactory/api/pypi/pypi-remote/simple

Related part of code:

https://github.com/frostming/unearth/blob/7c66e3d6b89f09814caa92f82767e0905156b8d6/src/unearth/auth.py#L194-L202

Can you provide the exact version of unearth running in your environment?

[racc]

It's running unearth 0.8.1

[racc]

I found the issue in the PdmBasicAuth class and created a PR to address it:
#1759

Please review