Skip to content

Commit 9da919e

Browse files
docs: fix incorrect reference to LLM in NonLLMContextPrecisionWithReference metric (#1863)
Updated the documentation for NonLLMContextPrecisionWithReference to clarify that it uses non-LLM-based similarity measures, correcting the misleading reference to LLMs. fixes #1861
1 parent 064ceed commit 9da919e

File tree

3 files changed

+4
-4
lines changed

3 files changed

+4
-4
lines changed

docs/concepts/metrics/available_metrics/context_precision.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -68,11 +68,11 @@ Output
6868

6969
## Non LLM Based Context Precision
7070

71-
The following metrics uses traditional methods to identify if a retrieved context is relevant or not. You can use any non LLM based metrics as distance measure to identify if a retrieved context is relevant or not.
71+
This metric uses traditional methods to determine whether a retrieved context is relevant. It relies on non-LLM-based metrics as a distance measure to evaluate the relevance of retrieved contexts.
7272

7373
### Context Precision with reference contexts
7474

75-
`NonLLMContextPrecisionWithReference` metric is can be used when you have both retrieved contexts and also reference contexts associated with a `user_input`. To estimate if a retrieved contexts is relevant or not this method uses the LLM to compare each of the retrieved context or chunk present in `retrieved_contexts` with each ones present in `reference_contexts`.
75+
The `NonLLMContextPrecisionWithReference` metric is designed for scenarios where both retrieved contexts and reference contexts are available for a `user_input`. To determine if a retrieved context is relevant, this method compares each retrieved context or chunk in `retrieved_context`s with every context in `reference_contexts` using a non-LLM-based similarity measure.
7676

7777
#### Example
7878

src/ragas/testset/synthesizers/multi_hop/base.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ async def _generate_sample(
161161
self, scenario: Scenario, callbacks: Callbacks
162162
) -> SingleTurnSample:
163163
if not isinstance(scenario, MultiHopScenario):
164-
raise TypeError('scenario type should be MultiHopScenario')
164+
raise TypeError("scenario type should be MultiHopScenario")
165165
reference_context = self.make_contexts(scenario)
166166
prompt_input = QueryConditions(
167167
persona=scenario.persona,

src/ragas/testset/synthesizers/single_hop/base.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ async def _generate_sample(
122122
self, scenario: Scenario, callbacks: Callbacks
123123
) -> SingleTurnSample:
124124
if not isinstance(scenario, SingleHopScenario):
125-
raise TypeError('scenario type should be SingleHopScenario')
125+
raise TypeError("scenario type should be SingleHopScenario")
126126
reference_context = scenario.nodes[0].properties.get("page_content", "")
127127
prompt_input = QueryCondition(
128128
persona=scenario.persona,

0 commit comments

Comments
 (0)