REFACTOR: De-dup scorer output_key defaults onto Scorer base as ClassVars - #1970
Closed
Roman Lutz (romanlutz) wants to merge 1 commit into
Closed
Conversation
…Vars Promote the five repeated `*_output_key` string defaults (`score_value`, `rationale`, `description`, `metadata`, `category`) onto the `Scorer` base as `DEFAULT_*_OUTPUT_KEY: ClassVar[str]`. `Scorer._score_value_with_llm_async`, `FloatScaleScorer`, `SelfAskGeneralFloatScaleScorer`, and `SelfAskGeneralTrueFalseScorer` now use the sentinel pattern (`str | None = None` + resolve to the class default) instead of duplicating the literals. No value changes. No behaviour changes. Verified: ruff / ty / pytest tests/unit/score/ (1184 passed, 16 skipped). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (rlundeen2)
approved these changes
Jun 9, 2026
Contributor
Author
|
Closing per maintainer feedback on the constants-audit work. This PR introduced the sentinel-default pattern (changing Branch left intact in case any portion is worth cherry-picking later. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
REFACTOR: De-dup scorer
output_keydefaults ontoScorerbase asClassVarsScope
Part of the constants-audit cleanup (see PRs #1964 and #1965 for the established pattern). PR B-2 of the staged plan.
What changed
The same five string defaults for the LLM scoring JSON-response key names were duplicated across four sibling classes:
score_value_output_key = "score_value"rationale_output_key = "rationale"description_output_key = "description"metadata_output_key = "metadata"category_output_key = "category"They are now declared once on the
Scorerbase asClassVar[str]:Touched files:
pyrit/score/scorer.py— promote constants;_score_value_with_llm_asyncnow uses the sentinel pattern (str | None = None, resolved toself.DEFAULT_*_OUTPUT_KEY).pyrit/score/float_scale/float_scale_scorer.py— override signature widened tostr | None = None; the override forwards values through tosuper()which resolves the sentinel.pyrit/score/float_scale/self_ask_general_float_scale_scorer.py— constructor now uses the sentinel pattern;self._*_output_keyis resolved from the class default when no value is provided.pyrit/score/true_false/self_ask_general_true_false_scorer.py— same constructor change.Behavioural compatibility
SelfAskGeneralTrueFalseScorer()(no kwargs) still producesself._score_value_output_key == "score_value", etc.SelfAskGeneralTrueFalseScorer(score_value_output_key="custom")still producesself._score_value_output_key == "custom"._score_value_with_llm_asyncthat omit these args (e.g.SelfAskTrueFalseScorer,SelfAskRefusalScorer,SelfAskQuestionAnswerScorer,SelfAskCategoryScorer,SelfAskScaleScorer,SelfAskLikertScorer,InsecureCodeScorer) get the same resolved defaults as before.No value changes. No behaviour changes.
Verified
uv run --link-mode=copy ruff check pyrit/score/— clean.uv run --link-mode=copy ty check pyrit/score/— no new diagnostics (one pre-existingcv2optional-import warning invideo_scorer.py).uv run --link-mode=copy pytest tests/unit/score/ -x --no-header -q—1184 passed, 16 skipped.