-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add hooks for calling arbitrary code around each benchmark #193
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the concept of hooks.
Would it be possible to use importlib.metadata instead of setuptools pkg_resources? See: https://docs.python.org/dev/library/importlib.metadata.html#entry-points
Sure -- I've never used entry points before -- I hadn't realized that's the new canonical way. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM overall. I don't know entry points, so I cannot review this change in details.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This change adds a concept of "hooks" to pyperf for running arbitrary code before and after the actual internal benchmarking code. These are implemented with entry points, so it should be possible to write an external plugin to add new integrations and functionality.
There are a few real use cases that the Faster CPython team already needs this for:
sys.monitoring
hooks around benchmarks to do custom analysis of which specific bytecodes are run during benchmarkingThe
pystats
integration that already exists in pyperf also should have, in hindsight, been implemented this way, so this PR also refactors things to do that, and it serves as a proof-of-concept of the new hook functionality.This PR needs tests but I thought I would submit a draft first to ensure this seems like the right direction first.