Skip to content

[Bug] Mirror issue-discovery caches a fabricated zero score when scoring_data_stored=False #836

Description

@taillred

Summary

In _resolve_solving_pr_score (gittensor/validator/issue_discovery/mirror_scan.py:418), a mirror response with scoring_data_stored=False is treated as a successful fetch of an empty diff, producing base_score=0 / token_score=0, which is then written into the cross-miner solving-PR cache. The cached zero is served to other miners in the same validator cycle whose discovered issues reference the same solving PR.

This is data-availability noise, not a real score — it should not be cached and should not consume the in-cycle retry budget.

Where the bug lives

gittensor/validator/issue_discovery/mirror_scan.py:438-455:

cache_stats.misses += 1
try:
    files_response = client.get_pr_files(issue.repo_full_name, solving_pr.pr_number)
except MirrorRequestError as e:
    cache_stats.fetch_failures += 1
    bt.logging.warning(...)
    return None

# No check on files_response.scoring_data_stored — falls straight through.
file_changes, file_contents = mirror_files_to_legacy(
    issue.repo_full_name, solving_pr.pr_number, files_response.files
)
result = calculate_base_score_for_pr_files(file_changes, file_contents, programming_languages, token_config)
cached = CachedSolvingPR(base_score=result.base_score, token_score=result.token_score)
cache[key] = cached
return cached

When scoring_data_stored=False, files_response.files is empty, the helper returns base_score=0 / token_score=0, and that zero is committed to the per-cycle cache.

Observable effects

  1. Cache stats are wrong. A scoring_data_stored=False response is counted as a successful fetch; cache_stats.fetch_failures does not increment. The end-of-phase log (gittensor/validator/issue_discovery/mirror_scan.py:198-201) under-reports mirror flakiness.
  2. No in-cycle retry. Once cached, sibling miners' lookups against the same (repo, pr_number) skip the network entirely and reuse the fabricated zero — even if the mirror finishes its backfill mid-cycle.
  3. Misleading per-issue logs. The same issue logs the "below token threshold — credibility only" branch (mirror_scan.py:353) instead of the "score unavailable — credibility only" branch (mirror_scan.py:316), conflating "we know it's low" with "we have no information."

Expected behavior

A scoring_data_stored=False response should be treated as an availability failure, identical to a MirrorRequestError:

  • Increment cache_stats.fetch_failures.
  • Log a warning that mentions data unavailability (not "below threshold").
  • Return None.
  • Do not write to cache — leaves the slot open for a later miner to retry within the cycle.

Acceptance

  • _resolve_solving_pr_score returns None and does not mutate cache when files_response.scoring_data_stored is False.
  • cache_stats.fetch_failures increments on the unavailable path.
  • Regression test: a single scoring_data_stored=False response feeding two issues across two miners results in two cache misses, two fetch-failure increments, and an empty cache.
  • No change to mirror OSS scoring (oss_contributions/mirror/scoring.py) and no change to credibility/eligibility behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions