Skip to content

reward: normalize_math_answer leaves a trailing backslash on LaTeX-escaped set braces \{...\} #162

Description

@philluiz2323

Bug

normalize_math_answer in src/trinity/orchestration/reward.py strips a single outer pair of { } / \{ \} with a greedy capture group:

# Strip a single outer pair of \{ \} or { }.
s = re.sub(r"^\\?\{(.*)\\?\}$", r"\1", s).strip()

For LaTeX-escaped set braces \{ … \}, the greedy (.*) swallows the closing backslash and the optional \\? matches nothing, leaving a stray trailing backslash in the normalized answer.

Reproduce (offline, no network/GPU)

from trinity.orchestration.reward import normalize_math_answer, math_equal
normalize_math_answer(r'\{5\}')       # -> '5\\'    (should be '5')
normalize_math_answer(r'\{1,2,3\}')   # -> '1,2,3\\'(should be '1,2,3')
math_equal(r'\{1,2,3\}', '1,2,3')     # -> False    (should be True)
normalize_math_answer('{5}')          # -> '5'      (unescaped case is fine)

Impact

Any math item whose gold answer or model output formats a set with escaped braces (\{a,b,c\}, common in MATH-style solutions) is graded as a false negative — the two sides are equal but compare unequal because one carries a trailing backslash. This silently understates accuracy on set-valued answers.

Expected

The escaped-brace pair should be stripped cleanly: \{1,2,3\}1,2,3.

Fix

Make the inner capture non-greedy so the optional trailing \\? can claim the escaping backslash:

s = re.sub(r"^\\?\{(.*?)\\?\}$", r"\1", s).strip()

Location

src/trinity/orchestration/reward.py, normalize_math_answer (the "Strip a single outer pair" line, ~819).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions