Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tooltip text for Scorers display #521

Open
david-fisher opened this issue May 27, 2022 · 3 comments · May be fixed by #556
Open

Tooltip text for Scorers display #521

david-fisher opened this issue May 27, 2022 · 3 comments · May be fixed by #556
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@david-fisher
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Scorers view could use documentation of the scorers to make it easier for users to choose which one to use.

Describe the solution you'd like
Write a one sentence blurb for each scorer, to be displayed as a tooltip.
Describe alternatives you've considered
none
Additional context
none

@david-fisher david-fisher added documentation Improvements or additions to documentation enhancement New feature or request labels May 27, 2022
@david-fisher david-fisher self-assigned this May 27, 2022
@david-fisher
Copy link
Contributor Author

david-fisher commented Jul 26, 2022

I need to do some research to figure out how to have tooltips in Quepid's UI, this text spans the expected tooltip content.

The metrics are:

These three are gain based metrics. Each new relevant document contributes some value when it appears in the ranked list.

nDCG@10 -- Normalized discounted sum of the gain values. The ratio of the DCG to the Ideal DCG that results from the optimal ordering of the rated relevant documents. Useful for comparing across queries. Useful for averaging over multiple queries.
DCG@10 -- Discounted sum of the gain values. Not useful for comparing across queries. Not useful for averaging over multiple queries.
CG@10 -- Simple sum of the gain values. Not useful for comparing across queries. Not useful for averaging over multiple queries.

P@10 -- The effective accuracy, of the top k (10) retrieved, how many of them were in the set of documents rated relevant. Useful for comparing across queries. Useful for averaging across queries.

AP (while still called @10, AP is computed on |R| the set of all relevant documents) The average of the precision points (accuracy) at each rank at which a relevant document appears. This average is across the full set of relevant documents. Useful for comparing across queries. Useful for averaging over multiple queries. This metric is the most common summary metric for capturing overall performance of a retrieval system independent of the retrieval task.

RR (MRR when averaged across queries), not yet merged into Quepid. The reciprocal of the rank of the first relevant document to occur in the ranked list. This metric is appropriate for evaluating known item, or single target item search. Somewhat useful for comparing across queries. Useful for averaging across queries.

v1-520 -- average rating 100 discounted by edit distance from best @ 10 (why is this in quepid?)
v1-527 -- is exactly the same code as v1-520 and should be removed as redundant.

@epugh epugh linked a pull request Aug 29, 2022 that will close this issue
@stale
Copy link

stale bot commented Apr 2, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please do advocate for this issue and ideally submit a patch to get the attention of the maintainers!

@stale stale bot added the wontfix This will not be worked on label Apr 2, 2023
@stale stale bot closed this as completed Aug 12, 2023
@epugh
Copy link
Member

epugh commented Aug 15, 2023

bad stale bot! We still want this! Sigh, better get this done someday ;-)

@epugh epugh reopened this Aug 15, 2023
@stale stale bot removed the wontfix This will not be worked on label Aug 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants