Skip to content

Commit 85979d0

Browse files
authored
fix lm_eval version issue (#621)
The current version of lm_eval as installed using `pip install -e ".[evolve]"` in the READ.ME file, no longer use the evaluator class. The code cannot run without this modification.
1 parent 5b67d01 commit 85979d0

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

mergekit/evo/helpers.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ def _eval_model(
3030
task_manager: Optional[lm_eval.tasks.TaskManager] = None,
3131
**kwargs,
3232
) -> Dict[str, Any]:
33-
results = lm_eval.evaluator.simple_evaluate(
33+
results = lm_eval.simple_evaluate(
3434
model=model,
3535
model_args=model_args,
3636
tasks=list(set([task.name for task in tasks])),

0 commit comments

Comments
 (0)