[Groudnedness] Handle strings and single entries #4621
Conversation
Test Results for assets-test0 tests 0 ✅ 0s ⏱️ Results for commit 1d0648f. ♻️ This comment has been updated with latest results. |
|
|
||
| if (not query) or (not response): # or not tool_definitions: | ||
| msg = ( | ||
| f"{type(self).__name__}: Either 'conversation' or individual inputs must be provided. " |
There was a problem hiding this comment.
Consider improving the error message. Every time I see this, I don't know which fields is missing.
There was a problem hiding this comment.
This "For Agent groundedness 'query' and 'response' are required." is specific to missing query/response. Do you mean to have specific one for query and another for response? Or remove the first generic part in the message Either 'conversation' or individual inputs must be provided.
| self, | ||
| *, | ||
| query: str, | ||
| response: List[dict], |
There was a problem hiding this comment.
response doesn't take str like other agent evaluators.
Please look at details in Bug
There was a problem hiding this comment.
It is handled in the code, but I just updated the type hints too
ashaabansoliman
left a comment
ashaabansoliman
left a comment
There was a problem hiding this comment.
approved
This is to handle a regression reported in this issue
No description provided.