The evaluation of MBPP is 1-shot or 3-shot? #867
Unanswered
xiaoshengjun
asked this question in
Q&A
Replies: 1 comment
-
你好,你这边找到对于的MBPP-3shot数据集了吗? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
In https://rank.opencompass.org.cn/leaderboard-llm shows the evaluation of MBPP is 1-shot, When the mouse is placed on the test score of MBPP。But in code configs/datasets/mbpp/mbpp_gen_1e1056.py, 'mbpp_infer_cfg' config file shows 3-shot. So 1-shot or 3-shot, which is used for MBPP?
Beta Was this translation helpful? Give feedback.
All reactions