Skip to content

MME_score.py cannot work #14

@dggeed

Description

@dggeed

Hi! The script MME_score.py cannot work properly and there is a bug in the code. For example, in the judgment

HALC/eval/MME_score.py

Lines 124 to 127 in fc32840

if gt_question.lower() in pred_caption:
pred_ans = True
else:
pred_ans = False

the pred_caption is a single word (i.e. Yes or No), whereas gt_question.lower() is a sentence. As a result, pred_ans = False almost always holds true. BTW, Can you add comments to the key parts of the code? Thanks a lot! :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions