Skip to content

REFACTOR: Promote multi-turn attack constants & defaults to ClassVar - #1968

Closed
Roman Lutz (romanlutz) wants to merge 1 commit into
microsoft:mainfrom
romanlutz:romanlutz/refactor-multiturn-attack-constants
Closed

REFACTOR: Promote multi-turn attack constants & defaults to ClassVar#1968
Roman Lutz (romanlutz) wants to merge 1 commit into
microsoft:mainfrom
romanlutz:romanlutz/refactor-multiturn-attack-constants

Conversation

@romanlutz

Copy link
Copy Markdown
Contributor

Continues the constants-audit cleanup established by #1964 (backend / registry / setup) and #1965 (prompt_converter). This PR combines two tracks for the multi-turn attacks so reviewers see the full picture per attack class:

  • Track A: promote the module-level _ADVERSARIAL_REQUIREMENTS TargetRequirements sentinel onto the owning attack class as a ClassVar.
  • Track B: lift hard-coded constructor defaults onto the owning class as DEFAULT_* ClassVars, using the sentinel-default pattern (x: int | None = None, then if x is None: x = self.DEFAULT_X) so values remain inheritable/overridable and stay self-documenting.

Files

  • pyrit/executor/attack/multi_turn/crescendo.py

    • _ADVERSARIAL_REQUIREMENTSCrescendoAttack._ADVERSARIAL_REQUIREMENTS: ClassVar[TargetRequirements]
    • max_backtracks: int = 10DEFAULT_MAX_BACKTRACKS: ClassVar[int] = 10
    • max_turns: int = 10DEFAULT_MAX_TURNS: ClassVar[int] = 10
  • pyrit/executor/attack/multi_turn/red_teaming.py

    • _ADVERSARIAL_REQUIREMENTSRedTeamingAttack._ADVERSARIAL_REQUIREMENTS: ClassVar[TargetRequirements]
    • max_turns: int = 10DEFAULT_MAX_TURNS: ClassVar[int] = 10
  • pyrit/executor/attack/multi_turn/tree_of_attacks.py

    • _ADVERSARIAL_REQUIREMENTSTreeOfAttacksWithPruningAttack._ADVERSARIAL_REQUIREMENTS: ClassVar[TargetRequirements]
    • tree_width: int = 3DEFAULT_TREE_WIDTH: ClassVar[int] = 3
    • tree_depth: int = 5DEFAULT_TREE_DEPTH: ClassVar[int] = 5
    • branching_factor: int = 2DEFAULT_BRANCHING_FACTOR: ClassVar[int] = 2
    • desired_response_prefix: str = "Sure, here is"DEFAULT_DESIRED_RESPONSE_PREFIX: ClassVar[str] = "Sure, here is"
    • batch_size: int = 10DEFAULT_BATCH_SIZE: ClassVar[int] = 10
  • pyrit/executor/attack/multi_turn/pair.py (Track B only — no module-level _ADVERSARIAL_REQUIREMENTS existed here)

    • tree_width: int = 3PAIRAttack.DEFAULT_TREE_WIDTH: ClassVar[int] = 3
    • tree_depth: int = 5PAIRAttack.DEFAULT_TREE_DEPTH: ClassVar[int] = 5
    • desired_response_prefix: str = "Sure, here is"PAIRAttack.DEFAULT_DESIRED_RESPONSE_PREFIX: ClassVar[str] = "Sure, here is"
    • batch_size: int = 10PAIRAttack.DEFAULT_BATCH_SIZE: ClassVar[int] = 10

Docstrings updated to point at the DEFAULT_* symbols (e.g. tree_width: defaults to ``DEFAULT_TREE_WIDTH`` (3)). Public call signatures stay keyword-only with the same parameter names; only the defaults moved from inline literals to class attributes.

No value or behaviour changes

Pure structural rename. All literals preserved; same defaults applied via the sentinel pattern. Validation order is preserved (is None substitution runs before existing positive/non-negative range checks).

Verified

uv run --link-mode=copy ruff check pyrit/executor/attack/multi_turn/
uv run --link-mode=copy ty check pyrit/executor/attack/multi_turn/
uv run --link-mode=copy pytest tests/unit/executor/attack/multi_turn/ -x --no-header -q

All green — 319 passed in the multi_turn test suite, ruff/ty clean.

Continues the constants-audit cleanup (microsoft#1964, microsoft#1965) by promoting the
module-level _ADVERSARIAL_REQUIREMENTS sentinels and hard-coded constructor
defaults (max_turns, max_backtracks, tree_width, tree_depth, branching_factor,
desired_response_prefix, batch_size) in crescendo, red_teaming, tree_of_attacks
and pair onto the owning attack classes as ClassVars, using the
sentinel-default pattern for hyperparameters so docstrings reference the
DEFAULT_* symbols.

No value changes. No behaviour changes.

Verified: ruff / ty / pytest tests/unit/executor/attack/multi_turn/.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@romanlutz

Copy link
Copy Markdown
Contributor Author

Closing per maintainer feedback on the constants-audit work.

This PR mixed two concerns: (1) the A-4 portion — promoting the module-level _ADVERSARIAL_REQUIREMENTS to a class-level ClassVar on each multi-turn attack — which matches the desired pattern, and (2) the B-3 portion — promoting hyperparameter defaults (max_turns, max_backtracks, tree_width, etc.) to DEFAULT_* ClassVars via the sentinel-default pattern (int = 10int | None = None). The latter widens public __init__ signatures, which is not desired.

Branch romanlutz/refactor-multiturn-attack-constants left intact; the A-4 portion can be re-extracted into a clean PR if you want it merged separately. Pure A-track PRs (#1964, #1965, #1972, #1973, #1974, #1975) remain open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants