fix(miner): retry transient 5xx/rate-limit in contribution-profile getJson#7126
Conversation
…tJson extractContributionProfile's getJson fetched a repo's label taxonomy and CONTRIBUTING.md by calling fetchImpl directly with no retry, so a single transient GitHub 5xx or rate-limit response (429 / secondary-403) degraded the signal to absent/unknown. Because discover-cli persists whatever extract() returns into the contribution-profile cache, that one blip got cached as the repo's eligibility signal until the entry expired, silently weakening label filtering for a repo that was reachable moments before or after. Route getJson through fetchWithRetry (./http-retry.js) — the same helper opportunity-fanout.js's sibling githubGetJson already uses — so a retryable status rides out its bounded attempts with exponential-backoff-or-Retry-After delay before falling back. The per-attempt REQUEST_TIMEOUT_MS is preserved via fetchWithRetry's timeoutMs (a fresh AbortSignal.timeout per attempt), the never-throws / fail-open contract is unchanged once retries are exhausted, and a sleepFn seam is threaded through so the retry is testable without real timers. Closes JSONbored#7090
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7126 +/- ##
=======================================
Coverage 95.98% 95.98%
=======================================
Files 609 609
Lines 48153 48154 +1
Branches 15133 15133
=======================================
+ Hits 46218 46219 +1
Misses 1118 1118
Partials 817 817
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
Tip ✅ LoopOver review result - approve/merge recommendedReview updated: 2026-07-17 22:41:04 UTC
Review summary Nits — 4 non-blocking
Decision drivers
Context & advisory signals — never blocks the verdict
Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
What & why
extractContributionProfile'sgetJson(
packages/loopover-miner/lib/contribution-profile-extract.js) fetched a repo's label taxonomy andCONTRIBUTING.mdby callingfetchImpldirectly with no retry, unlike its siblingopportunity-fanout.js'sgithubGetJson, which already wraps the identical class of request infetchWithRetry. A single transient GitHub 5xx or rate-limit response (429 / secondary-403) thereforedegraded the label/doc signal to
absent/unknownon the first blip.That matters more than a typical missing-retry gap:
discover-cli.js'sresolveContributionProfilesForDiscoverimmediatelycache.put()s whateverextract()returns into thepersistent contribution-profile cache, keyed by repo with its own TTL. So one transient 5xx during a
discoverrun got cached as the repo's eligibility signal and kept silently weakening label-basedfiltering until the entry expired — even though the repo's real label taxonomy was reachable moments
before or after the blip.
Change
getJsonnow routes throughfetchWithRetry(./http-retry.js) — same retryable-status semantics,bounded attempts, and exponential-backoff-or-
Retry-Afterdelay asgithubGetJson.fetchWithRetry'stimeoutMsoption (a freshAbortSignal.timeoutper attempt) — a straight pass-through, not a redesign.error, an unparseable body, or genuinely-exhausted retries all still return
null.sleepFnseam is threaded fromextractContributionProfile's options throughgetJson/fetchContributing, mirroring every otherfetchWithRetrycall site, so the retry is testable with noreal timers.
Tests
Added to
test/unit/contribution-profile-extract.test.ts(injectedsleepFn, no real timers):profile as an immediate success (not a degraded
absent).CONTRIBUTING.mdfetch returns a transient 5xx then succeeds → linked-issue rule read asexplicit,as if it never blipped.
absent, exactlyDEFAULT_MAX_ATTEMPTSattempts with 2 sleeps between them, never throws.
prBodystillabsent, fail-open contract preserved.100% Codecov patch coverage on every changed line and branch in
contribution-profile-extract.js(measured unsharded); the file's only non-covered branch is the pre-existing
githubHeadersno-token arm, outside this diff.
Closes #7090