Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterize the top_k_categorical_accuracy metric #172

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

somnathrakshit
Copy link

Modify to have parameterized version of accuracy metric

@jstypka
Copy link
Collaborator

jstypka commented Dec 26, 2018

Hey @somnathrakshit, thanks for the PR!

This change makes parametrisation of the top_k_accuracy metric possible, but doesn't enable us to do anything else. The way I thought about it, is to have the top_custom_categorical_accuracy function in the user code, outside of Magpie and enable a field called metrics=[] in the Magpie API to pass arbitrary functions to be computed. That way users can compute whatever they want. We should also enable some sensible defaults to be computed when nothing is passed in.

Hope that makes sense!

@somnathrakshit
Copy link
Author

Yes, it does. I will try to do it over the weekend but not sure if I will get time or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants