Skip to content

Fix EVMbench medium findings subdirectory option - #155

Open
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix-evmbench-medium-findings-subdir
Open

Fix EVMbench medium findings subdirectory option#155
sylvesterkaczmarek wants to merge 1 commit into
openai:mainfrom
sylvesterkaczmarek:fix-evmbench-medium-findings-subdir

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

  • align the EVMbench findings_subdir option with the audit-layer spelling
  • expose medium rather than med for the medium incorrectness dataset
  • leave the separate hint_level="med" option unchanged

The eval currently declares findings_subdir as "med", while Audit, Vulnerability, AuditRegistry, and the gold-audit utility all use "medium". Passing the eval's advertised medium value downstream therefore selects the wrong findings subdirectory.

This one-line change makes the eval configuration use the same "medium" convention as the rest of EVMbench.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant