[devgprime]

Watsonx continuation scoring compares the last token of the entire response with the last context token. For a non-empty continuation, this can incorrectly shorten the context by one token, include a prompt log probability in the answer score, and change the greedy-match result.

Compare at the context boundary instead, including the empty-context and tokenizer-added stop-token cases. Add ten offline regression cases for likelihood, greedy matching, empty continuations, and prefix mismatches. Four existing exception re-raises in the touched module use bare raise to satisfy the current pre-commit rules.

A minimal reproduction on main (d6de8164):

from lm_eval.models.ibm_watsonx_ai import WatsonxLLM

model = WatsonxLLM.__new__(WatsonxLLM)
tokens = [
    {"text": "prompt", "logprob": -9.0, "rank": 2},
    {"text": " answer", "logprob": -0.25, "rank": 1},
]
print(model._get_log_likelihood(tokens, ["prompt"]))

Before: LogLikelihoodResult(log_likelihood=-9.25, is_greedy=False).
After: LogLikelihoodResult(log_likelihood=-0.25, is_greedy=True).

Validation: the new test file has 5 failures and 5 passes on unchanged main, and all 10 cases pass with this fix. pytest tests/models/test_watsonx_loglikelihood.py -q and the repository's pre-commit hooks pass. Tests use recorded-response-shaped inputs without an SDK client, API credentials, network access or model downloads. Live Watsonx integration and the full model test suite were not run.

Related to #1613.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[CLAassistant]

All committers have signed the CLA.