fix(validator): handle unavailable mirror scoring data - #837
Conversation
|
@anderdc can you review this please? |
5266124 to
dc49509
Compare
YB0y
changed the title
Treat scoring_data_stored=False as an availability signal instead of a valid zero-score result in both mirror scoring paths. For issue discovery, solving PR file data that is unavailable now returns None and stays out of the per-cycle cache so later lookups can retry instead of hitting a cached zero. For OSS contributions, merged mirror PRs that cannot be scored are removed before finalization so they cannot inflate credibility or rescue reward eligibility. Closes entrius#836 Closes entrius#904
dc49509 to
c76b445
Compare
|
Hello @anderdc how are you? Could you review this please? |
|
Please stop pinging me incessantly. I will ignore your PRs if you continue that behavior or implement them myself. |
anderdc
left a comment
anderdc
left a comment
There was a problem hiding this comment.
Please scope this PR down to just the mirror_scan.py changes (the issue-discovery cache fix). That half is unambiguous correctness — caching an unavailable lookup as zero is wrong regardless of policy.
Drop the mirror/scoring.py changes (filtering unscored merged PRs out of mirror_eval.merged_prs). After cross-referencing with legacy at oss_contributions/scoring.py:86-91, this is actually a policy change rather than a bug fix — legacy leaves unscored PRs in the eval with no base_score, contributing 0 to score totals while still counting toward credibility, and we're keeping that behavior on mirror for parity. See the comment on #836 for full reasoning.
Drop the tests/.../test_scoring_data_stored_eligibility.py file too since it's tied to that half. Keep the new tests in test_mirror_scan.py.
|
No commits in 12+ hours since changes were requested. Closing. |
2 participants
Summary
Closes #836.
This PR treats the mirror's
scoring_data_stored=Falsesignal as unavailable scoring data, not as a valid zero-score result. It covers two sibling bugs in different validator paths:_resolve_solving_pr_scorenow returnsNoneand leaves the per-cycle solving-PR cache untouched when mirror file scoring data is unavailable. This prevents one unavailable response from poisoning later miners' issue-discovery scoring with a cached zero.Fix
MirrorPullRequestFilesResponse.scoring_data_storedafter fetching solving-PR files ingittensor/validator/issue_discovery/mirror_scan.py.Falsefromscore_mirror_prwhen a mirror PR cannot be scored because data is unavailable, file fetching fails, or no files are returned.MERGEDmirror PRs inmirror_eval.merged_prsbeforefinalize_miner_scoresuses those PRs for eligibility, credibility, and score aggregation.testand squashed into one commit.Test plan
uv run --extra dev python -m pytest tests/validator/issue_discovery/test_mirror_scan.py tests/validator/oss_contributions/mirror -quv run --extra dev ruff check gittensor/validator/issue_discovery/mirror_scan.py gittensor/validator/oss_contributions/mirror/scoring.py tests/validator/issue_discovery/test_mirror_scan.py tests/validator/oss_contributions/mirror/test_scoring_data_stored_eligibility.pyuv run --extra dev ruff format --check gittensor/validator/issue_discovery/mirror_scan.py gittensor/validator/oss_contributions/mirror/scoring.py tests/validator/issue_discovery/test_mirror_scan.py tests/validator/oss_contributions/mirror/test_scoring_data_stored_eligibility.py