FR: Proof of Performance Claims in Blog

[adam-grant-hendry]

Is your feature request related to a problem? Please describe.

I am finding pdm performance to be much slower than poetry on average, especially when adding or removing packages. Each time, pdm runs through the entire dependency list re-resolving pins of packages.

Your blog post make a claim that it is faster than poetry and pipenv, but tests only a single package: click.

1. What are the numbers? (seconds?, minutes?, relative units?)
2. What hardware is the system tested on?
3. How many runs were performed? Is this a single run or an average?

I'm finding on Windows, at least for my project, that pdm is slower than the alternatives and just as prone to bugs and errors.

Describe the solution you'd like

Please provide more definitive proof that pdm is indeed faster than pipenv and poetry. I am not finding this to be the case.

[frostming]

It is posted in the documentation, the results may vary depending on the dependencies set you choose. Maybe I am lucky that the chosen dependencies just work better on PDM by coincidence?

You can run the benchmark locally by pdm run benchmark. You need to pre-install the package managers.

[frostming]

but tests only a single package: click

@adam-grant-hendry It is not true. In the post, I added click to an existing dependency set, if you read carefully. I use this method to demonstrate only PDM reuses the existing pins and a little work is required to add a simple package.

[frostming]

FWIW, there was a defect in resolution, and the fix hasn't been released, you can try the main branch if you find any possible resolution loops. If you find the posted result is outdated I will update it.

[adam-grant-hendry]

@frostming If you find the posted results are inaccurate on your blog, I would highly recommend correcting them, yes. That is the ethical thing to do.

[frostming]

I made another run with the latest code: https://github.com/pdm-project/pdm/runs/6855138688?check_suite_focus=true

I think the conclusion still applies, as the time of this comment, Poetry lock is still running after an insanely long time. If you find PDM slow, you should better file an issue with the dependency set you are using.