Skip to content

Commit

Permalink
clean up judge settings example
Browse files Browse the repository at this point in the history
  • Loading branch information
rchan26 committed Sep 11, 2024
1 parent a416a2b commit 29a50c3
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions docs/evaluation.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,12 @@ Therefore, given a _completed_ experiment file (i.e., a jsonl file where each li
To run an LLM-as-judge evaluation, you must first create a _judge folder_ consisting of:
```
└── judge_folder
└── settings.json: a dictionary where keys are judge identifiers and the values are also dictionaries containing the "api", "model_name", and "parameters" to specify the LLM to use as a judge.
└── settings.json: a dictionary where keys are judge identifiers
and the values are also dictionaries containing the "api",
"model_name", and "parameters" to specify the LLM to use as a judge.
...
└── template .txt files: several template files that specify how to generate the prompts for the judge evaluation
└── template .txt files: several template files that specify how to
generate the prompts for the judge evaluation
```

#### Judge settings file
Expand Down

0 comments on commit 29a50c3

Please sign in to comment.