why answer_relavancy getting nan and sometime get 0. why is that? #2326
Description
[ ] I checked the documentation and related resources and couldn't find an answer to my question.
Your Question
why answer_relavancy getting nan and sometime get 0. why is that?
Code Examples
from datasets import Dataset from ragas.metrics import ( faithfulness, answer_relevancy, context_precision, context_recall ) from ragas import evaluate
eval_data = [ { "question": "What are the most valuable features of Canva according to the participants?", "reference": ( '"The brand kit feature is essential for client work. We can set up each client\'s colors and fonts, and everyone on the team can access them instantly" - Jennifer Walsh (Current Customer); ' '"The stock photos were definitely the best feature. Really high-quality images that I could use for product backgrounds" - Marcus Rodriguez (Churned Customer); ' '"The API capabilities are crucial, though they could be more robust. For actual design work, the drag-and-drop editor is surprisingly intuitive" - David Kim (Partner); ' '"Templates are helpful too for getting started, but I always felt limited by them" - Marcus Rodriguez (Churned Customer); ' '"The collaboration tools are great too - clients can comment directly on designs instead of sending revision emails" - Jennifer Walsh (Current Customer)' ) } ]
`rows = []
for item in eval_data:
docs = vector_store.similarity_search(item["question"], k=10)
contexts = [doc.page_content for doc in docs]
rows.append({
"question": item["question"],
"answer": get_chatbot_response(item["question"], vector_store),
"reference": item["reference"],
"contexts": contexts
})`
`from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision, context_recall
results = evaluate(
dataset,
metrics=[faithfulness, answer_relevancy, context_precision, context_recall],
)
print(results)`
Output Result
