Skip to content

test(ci): add unit-test seam for ci-duration-report.mjs percentile/summarize - #7479

Closed
lourincedaging0-commits wants to merge 1 commit into
JSONbored:mainfrom
lourincedaging0-commits:chore/ci-duration-report-test-seam-7456
Closed

test(ci): add unit-test seam for ci-duration-report.mjs percentile/summarize#7479
lourincedaging0-commits wants to merge 1 commit into
JSONbored:mainfrom
lourincedaging0-commits:chore/ci-duration-report-test-seam-7456

Conversation

@lourincedaging0-commits

Copy link
Copy Markdown
Contributor

Closes #7456

What

scripts/ci-duration-report.mjs's percentile() and summarize() carry real, non-obvious behavior — percentile() is a nearest-rank formula with a floor clamp, and summarize() deliberately excludes cancelled runs from both the duration set and the failure-rate denominator (a cancel-in-progress re-push artifact, not a CI failure) while treating skipped as a success. Neither was exported or tested; both only ran inside the un-guarded top-level body, which also makes a live GitHub API call — so nothing was testable in isolation.

This PR:

  • Exports percentile, summarize, and durationSeconds as named exports.
  • Moves the env-reading + live-fetch + report-assembly driver into main() behind an entrypoint guard (import.meta.url === pathToFileURL(process.argv[1]).href), so importing the module for tests never triggers a fetch. Running the script directly is unchanged.
  • Adds test/unit/ci-duration-report-script.test.ts.

Tests (all pass locally under vitest run test/unit)

  • percentile at p50/p95 against a known sorted array, plus a single-element array and an empty array (null).
  • summarize excludes cancelled from count and the failure-rate denominator while reporting them as excludedCancelled; treats skipped as a success (not a failure); returns null percentiles/failureRate for an empty set.
  • durationSeconds = (updated_at − created_at) in seconds.

Verified importing the module with empty env neither throws nor fetches (exports resolve to just the pure functions). scripts/** isn't in coverage.include, so no numeric patch gate — the new test is real and green.

…mmarize

percentile() (nearest-rank with a floor clamp) and summarize() (excludes cancelled runs
from both the duration set and the failure-rate denominator; treats skipped as success)
carried real, non-obvious behavior but only ran inside the script's un-guarded top-level
body, which also makes a live GitHub API call -- untestable in isolation.

Export percentile/summarize/durationSeconds and move the env-reading, live-fetch, and
report-assembly driver behind an entrypoint guard (import.meta.url === argv[1]) so
importing the module for tests never fetches. Add test/unit/ci-duration-report-script.test.ts
covering percentile at p50/p95 (incl. single-element and empty->null), summarize excluding
cancelled from count + failure denominator (counting them as excludedCancelled), and
summarize treating skipped as a success. Behavior of the script when run directly is
unchanged.

Closes JSONbored#7456
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.36%. Comparing base (dca5d66) to head (16aa211).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7479   +/-   ##
=======================================
  Coverage   91.36%   91.36%           
=======================================
  Files         717      717           
  Lines       73022    73022           
  Branches    21629    21632    +3     
=======================================
  Hits        66719    66719           
  Misses       5265     5265           
  Partials     1038     1038           
Flag Coverage Δ
shard-1 34.42% <ø> (+0.25%) ⬆️
shard-2 39.00% <ø> (-0.31%) ⬇️
shard-3 36.38% <ø> (+3.78%) ⬆️
shard-4 39.14% <ø> (-0.39%) ⬇️
shard-5 32.51% <ø> (-4.08%) ⬇️
shard-6 34.96% <ø> (+0.63%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 20, 2026
@loopover-orb

loopover-orb Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Caution

🛑 LoopOver review result - fixes required

Review updated: 2026-07-20 12:08:30 UTC

2 files · 1 AI reviewer · no blockers · CI failing · blocked

🛑 Suggested Action - Fix Blockers

Review summary
This PR mechanically hoists the live-fetch/env-reading driver into a guarded main() and adds named exports for percentile/durationSeconds/summarize, enabling the new unit test file to exercise the nearest-rank percentile clamp, cancelled-run exclusion, and skipped-as-success logic in isolation without hitting the GitHub API. The refactor is a faithful move of pre-existing code (fetchAllRuns now takes repo/token as params instead of closing over them, which is the only real behavioral touch) and the entrypoint guard correctly gates on argv[1] via pathToFileURL. Tests are real and assert against the actual exported pure functions rather than fabricated payloads, and CI failures (validate/validate-code) are unrelated to logic correctness here since they aren't shown as content defects.

Nits — 5 non-blocking
  • The `24 * 60 * 60 * 1000` millisecond-conversion literals moved into main() (scripts/ci-duration-report.mjs:80) are unchanged pre-existing code, not new complexity introduced by this diff, so flagging them as magic numbers is low-value churn.
  • scripts/ci-duration-report.mjs's entrypoint guard uses `process.argv[1] ?? ""` — worth a one-line comment noting this is defensive for edge environments where argv[1] is undefined, since it's non-obvious why the fallback exists.
  • The PR description says 'Closes test(ci): ci-duration-report.mjs's percentile/summarize logic has no unit-test seam #7456' but the issue itself isn't visible in this context; worth confirming the issue text matches the stated seam/testability goal.
  • Consider also testing that `main()` throws when GITHUB_REPOSITORY/GITHUB_TOKEN are unset, though that would require exporting main() or spawning the script as a subprocess — reasonable to skip given scripts/** isn't coverage-gated.
  • The comment in ci-duration-report.mjs:64-68 duplicating the summarize() rationale in the test file (lines 8-11) is intentional documentation-mirroring; fine as-is but could reference the source comment instead of restating it to avoid drift.

CI checks failing

  • validate
  • validate-code

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #7456
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 100 registered-repo PR(s), 49 merged, 5 issue(s).
Contributor context ✅ Confirmed Gittensor contributor lourincedaging0-commits; Gittensor profile; 100 PR(s), 5 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff exports percentile, summarize, and durationSeconds, gates the fetch/report-assembly driver behind an import.meta.url entrypoint guard so importing the module no longer triggers a live fetch, and adds test/unit/ci-duration-report-script.test.ts covering p50/p95 (including single-element and empty arrays), cancelled-run exclusion from count/failureRate while tracking excludedCancelled, and

Review context
  • Author: lourincedaging0-commits
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: HTML
  • Official Gittensor activity: 100 PR(s), 5 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Triage stale or unlinked PRs.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

LoopOver is closing this pull request on the maintainer's behalf (CI is failing (validate, validate-code)). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@loopover-orb loopover-orb Bot closed this Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(ci): ci-duration-report.mjs's percentile/summarize logic has no unit-test seam

1 participant