Skip to content

Commit

Permalink
Add use_effective_order parameter
Browse files Browse the repository at this point in the history
Original BLEU fails at answers shorter than four tokens, since the amount of 4-grams is zero. Adding the use effective order parameter allows the user to still obtain scores considering only n-grams up to the amount of tokens in the answer when N<4.
  • Loading branch information
JonasElburgUVA committed Jan 21, 2025
1 parent 37dacd5 commit e796e01
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions src/ragas/metrics/_bleu_score.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ class BleuScore(SingleTurnMetric):
default_factory=lambda: {MetricType.SINGLE_TURN: {"reference", "response"}}
)
language: str = "english"
use_effective_order: bool=False

def __post_init__(self):
try:
Expand Down

0 comments on commit e796e01

Please sign in to comment.