You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
LLM judges, or "Synthetic Judge" as Doug Rosenoff refers to them, require some explanation for why a evaluation scored the way it did.
Today we only ask for explanations on a "I can't judge" response from a human judge. We need to also make it possible to get explanations from our jduges.
Describe the solution you'd like
Provide an opt int switch ont he judgement UI that flips on the explantions, and then leverages the keyboard to allow you to rate, and then tab to the explantion, and then tab to submit. Disables the keyboaard Rate and submit.
Describe alternatives you've considered
Thought about a required switch at the book level, but still need to figure out the workflow.
Is your feature request related to a problem? Please describe.
LLM judges, or "Synthetic Judge" as Doug Rosenoff refers to them, require some explanation for why a evaluation scored the way it did.
Today we only ask for explanations on a "I can't judge" response from a human judge. We need to also make it possible to get explanations from our jduges.
Describe the solution you'd like
Provide an opt int switch ont he judgement UI that flips on the explantions, and then leverages the keyboard to allow you to rate, and then tab to the explantion, and then tab to submit. Disables the keyboaard Rate and submit.
Describe alternatives you've considered
Thought about a required switch at the book level, but still need to figure out the workflow.
Additional context
Anything else @david-fisher @sstults ?
The text was updated successfully, but these errors were encountered: