FIX: stop seeded converters from reseeding the global RNG

[Vishnu Rajeev (VishnuR23)]

Description

Three components seed Python's process-wide RNG when given a seed:

• ZalgoConverter.validate_input — random.seed(self._seed)
• ProportionSelectionStrategy.select_range (anchor="random") — random.seed(self._seed)
• WordProportionSelectionStrategy.select_words — random.seed(self._seed)

Each then draws from the random module itself. Because validate_input / select_* run on every conversion, a single seeded instance resets global random state repeatedly, and roughly 17 modules under pyrit/ draw from that same global RNG — CharSwapConverter, RandomCapitalLettersConverter, InsertPunctuationConverter, EmojiConverter, LeetspeakConverter, UnicodeConfusableConverter, SeedDataset sampling, and others.

The result: seeding one converter for reproducibility silently de-randomizes unrelated converters in the same process. For a red-teaming framework this quietly costs attack diversity — a campaign keeps re-testing the same variations while appearing randomized.

Reproduction on main — an unrelated converter, alongside a seeded ZalgoConverter:

caps = RandomCapitalLettersConverter(percentage=50.0)
zalgo = ZalgoConverter(seed=42)          # unseeded in the control run
for _ in range(4):
    await zalgo.convert_async(prompt="hello")
    print((await caps.convert_async(prompt="the quick brown fox jumps")).output_text)

control (ZalgoConverter())          -> 4 distinct outputs
main    (ZalgoConverter(seed=42))   -> 1 distinct output, repeated 4x
    thE QuIcK BROwn Fox JumPs
    thE QuIcK BROwn Fox JumPs
    thE QuIcK BROwn Fox JumPs
    thE QuIcK BROwn Fox JumPs

Fix

Each of the three now owns a random.Random instance and reseeds that rather than the global module. random.Random(seed) yields the same sequence as random.seed(seed) plus the module-level functions, so seeded output is byte-identical to before — I verified this by capturing outputs for seeds 1/42/123 on both sides of the change and diffing them. Only the global side effect is removed. grep -rn "random\.seed(" pyrit/ is now empty.

Note this does change one edge case: an unseeded instance no longer inherits a user's global random.seed(...). That path is what the per-component seed argument is for, and relying on it is what caused the bug.

Tests and Documentation

Three regression tests assert random.getstate() is unchanged across a seeded call — precise and non-flaky, no reliance on sampling luck:

• test_zalgo_seed_does_not_disturb_global_rng
• TestProportionSelectionStrategy::test_select_range_seed_does_not_disturb_global_rng
• TestWordProportionSelectionStrategy::test_select_words_seed_does_not_disturb_global_rng

All three fail on main and pass with the fix (confirmed by reverting only the source changes and re-running: 3 failed, 103 passed). Also added test_zalgo_seed_is_repeatable_on_same_instance and test_zalgo_unseeded_converters_stay_independent to pin both directions of the contract.

One existing test needed updating: test_char_swap_converter_proportion_unchanged_with_iterations patched random.sample to control word selection, which worked only because the strategy called the global module. It now patches the strategy's own RNG; the assertion it exists for (selection happens once, not per iteration) is unchanged.

Verification:

• pytest -n 4 --dist=loadfile tests/unit -> 15130 passed, 121 skipped
• pytest tests/unit/converter -> 1125 passed, 34 skipped
• pre-commit run --files <changed> -> all hooks pass, including ruff format, ruff check, and ty

No documentation changes — internal RNG ownership only, no public API or notebook surface affected.

[Roman Lutz (romanlutz)]

This no longer guarantees the converter's documented reproducibility when Zalgo is composed with an unseeded random word selector. WordLevelConverter.convert_async() performs word selection after this reset, but WordProportionSelectionStrategy now owns an independent RNG, so repeated calls to ZalgoConverter(seed=42, word_selection_strategy=WordProportionSelectionStrategy(proportion=0.5)) can select different words and produce different outputs. Please either use one operation-local RNG across selection and mark generation, or explicitly define seeds as component-scoped and update the public contract accordingly. Add a regression test for this composed case.

[Roman Lutz (romanlutz)]

This regression test mutates the process-wide RNG and leaves it seeded at 0, which can make later tests order-dependent. The same pattern appears in both new selection-strategy tests. No setup seed is needed here: capture the existing random.getstate(), exercise the component, and compare against that state. Alternatively, restore the original state in a finally block.

[Vishnu Rajeev (VishnuR23)]

Vishnu Rajeev (Vishnu Rajeev (@VishnuR23)) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

[Vishnu Rajeev (VishnuR23)]

Thanks — both were right, and the second turned out to be more interesting than it looked. Pushed fixes for both.

1. Composed reproducibility (zalgo_converter.py)

Confirmed. ZalgoConverter(seed=42, word_selection_strategy=WordProportionSelectionStrategy(proportion=0.5)) gave 4 distinct outputs across 5 calls after my change, where main gave 1 — word selection had been riding on the global seed the converter set, which is exactly the coupling this PR removes.

I took your second option, seeds are component-scoped, rather than threading one operation-local RNG through selection and mark generation. Reasoning: the operation-local route means WordLevelConverter.convert_async has to build an RNG and pass it into WordSelectionStrategy.select_words, changing the strategy interface for all nine implementations — seven of which are deterministic and have no use for it. That is a wide API change to preserve one behavior that only ever worked as a side effect. Component-scoped seeds also match the "pluggable brick" framing in doc/code/framework.md: each component owns its randomness, and you seed the ones you want pinned. Both already take seed, so nothing new is needed:

ZalgoConverter(
    seed=42,
    word_selection_strategy=WordProportionSelectionStrategy(proportion=0.5, seed=7),
)   # -> identical output across calls

Contract updated on all three seed params, including the note that the default selection strategy takes every word and is deterministic, so seed alone still fully determines output in the common case. Added test_zalgo_seed_is_component_scoped_when_composed_with_random_selection covering both halves. Sized that prompt to 8 words (C(8,4)=70 selections) so the "varies" half cannot flake.

Happy to switch to the operation-local RNG if you would rather one seed control the whole pipeline — bigger diff, but I do not mind doing it.

2. Tests mutating global RNG (test_zalgo_converter.py)

Right, and thanks — leaving the global RNG seeded at 0 was the same class of bug this PR is about, in the tests.

One wrinkle worth flagging: your first suggestion (drop the setup seed, capture the existing state and compare) makes two of the three tests pass vacuously. Without a distinct setup seed, state_before is whatever the previous test left behind — and in TestProportionSelectionStrategy the preceding test seeds 42 and draws, which is exactly what the leaking code under test does. It reseeds to the same value, consumes the same numbers, and lands back on the captured state, so the assertion holds while the leak is real. Verified: with only that change, reverting the source fix left 107 passed instead of failing.

So I used your finally alternative, which removes the pollution without weakening the assertion: seed to a value distinct from the component's own seed, capture, exercise, assert, then random.setstate(original) in finally.

Verified both directions:

• All four regression tests fail against the pre-fix source (git checkout HEAD~1 -- pyrit/converter/...) and pass with the fix.
• The tests leave global state untouched — seeding to a known value, running each test in-process, then re-reading random.getstate() compares equal.
• pytest -n 4 --dist=loadfile tests/unit -> 15397 passed, 6 skipped. pre-commit clean, including ruff and ty.