Skip to content

feat(scoring): explain score scenario previews for miners#1935

Closed
andriypolanski wants to merge 2 commits into
JSONbored:mainfrom
andriypolanski:feat/scoring-explain-score-scenarios
Closed

feat(scoring): explain score scenario previews for miners#1935
andriypolanski wants to merge 2 commits into
JSONbored:mainfrom
andriypolanski:feat/scoring-explain-score-scenarios

Conversation

@andriypolanski

Copy link
Copy Markdown
Contributor

Summary

buildScorePreview (src/scoring/preview.ts) already emits rich what-if scenario previews (scenarioPreviews: cleanGates, afterPendingMerges, afterApprovedPrsMerge, afterStalePrsClose, linkedIssueFixed, bestReasonableCase) and gate deltas (gateDeltas) comparing current vs projected multiplier stacks. Miners consuming gittensory_preview_local_pr_score get raw structured data, and gittensory_explain_score_breakdown explains per-multiplier levers — but no tool narrates which scenario path unlocks scoreability or ranks the highest-leverage cleanup sequence (land pending merges vs fix linked issue vs clear open-PR pressure).

Contributors stuck at conditionally_scoreable see numeric deltas in JSON without plain-English guidance on the best next state transition.

What this adds

A pure projection service explainScoreScenarios mirroring explainScoreBreakdown:

  • Scenario cards for each non-current preview: name, source (github_observed / user_supplied / gittensory_projection), sanitized assumptions, scoreability band (blockedconditionally_scoreablescoreable), and a one-line delta vs current.
  • Gate delta narratives expanding each ScoreGateDelta into actionable copy (which gate moves, what changes, estimated direction — without leaking forbidden reward language).
  • Recommended path: single highest-leverage scenario + ordered levers (e.g. "close stale PRs before opening new work" when afterStalePrsClose is the top unlock).
  • MCP tool gittensory_explain_score_scenarios and matching API route — same input shape as gittensory_explain_score_breakdown / score preview.

Purely additive explanation over already-computed preview fields; no scoring behavior change.

Files touched (estimated)

File Change
src/services/score-scenario-explain.ts New: explainScoreScenarios, scenario ranking, gate-delta narratives
src/mcp/server.ts Register gittensory_explain_score_scenarios + output schema
src/api/routes.ts POST /v1/repos/:owner/:repo/score/scenario-explanation (or sibling to existing breakdown route)
src/openapi/schemas.ts ScoreScenarioExplanation schema
apps/gittensory-ui/public/openapi.json Regenerated via npm run ui:openapi
test/unit/score-scenario-explain.test.ts Blocked→conditional paths, pending-merge unlock, linked-issue fix, forbidden-term guard
test/unit/mcp-output-schemas.test.ts MCP structured output validation
test/integration/api.test.ts Route smoke + contributor scoping

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — aim for 100% statements/branches on changed lines in src/services/score-scenario-explain.ts
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate

Targeted test run:

npx vitest run test/unit/score-scenario-explain.test.ts test/unit/mcp-output-schemas.test.ts --coverage --coverage.include='src/services/score-scenario-explain.ts'

Notes

Analogues to imitate end-to-end: explainScoreBreakdown in src/services/score-breakdown.ts, renderPublicScenarioSummary / OPTION_NEXT_STEPS in src/scenarios/scenario-summary.ts, and MCP registration for gittensory_explain_score_breakdown in src/mcp/server.ts.

@andriypolanski andriypolanski requested a review from JSONbored as a code owner July 1, 2026 00:04
@dosubot dosubot Bot added the size:L label Jul 1, 2026
@andriypolanski andriypolanski changed the title Feat/scoring explain score scenarios feat(scoring): explain score scenario previews for miners Jul 1, 2026
@dosubot

dosubot Bot commented Jul 1, 2026

Copy link
Copy Markdown

📄 Knowledge review

✏️ Suggested updates

1 page suggestion needs review.

Page Library Status
MCP & Agent Integration gittensory 🟡 Review
📝 MCP & Agent Integration
@@ -10,18 +10,18 @@
 
 The **Model Context Protocol (MCP)** is a standardized interface that allows coding agents and AI tools to connect with external data sources and services in a structured, tool-call-based way. Rather than ad-hoc API integrations, MCP defines a common schema for tool registration, input validation, and structured JSON responses — enabling any MCP-compatible client (Claude Desktop, Cursor, Codex, etc.) to interact with a server like Gittensory without custom wiring.
 
-`@jsonbored/gittensory-mcp` implements MCP using `@modelcontextprotocol/sdk` v1.29.0 [[4]](https://app.dosu.dev/7d8810aa-91c6-4074-849b-92fab65de734/documents/09d20768-c6a2-4ac0-9000-4395a9728dc9#L323-L334). It registers 21 tools across five capability areas and exposes them over standard stdio transport, making Gittensory's intelligence layer accessible to any MCP-compatible client.
+`@jsonbored/gittensory-mcp` implements MCP using `@modelcontextprotocol/sdk` v1.29.0 [[4]](https://app.dosu.dev/7d8810aa-91c6-4074-849b-92fab65de734/documents/09d20768-c6a2-4ac0-9000-4395a9728dc9#L323-L334). It registers 22 tools across five capability areas and exposes them over standard stdio transport, making Gittensory's intelligence layer accessible to any MCP-compatible client.
 
 ### What it provides
 
 The package exposes two surfaces for different audiences:
 
-- **21 MCP tools** — structured tool definitions callable by AI agents via the MCP protocol, covering repo intelligence, PR preflight, scoring, contributor decisions, current-branch analysis, and agent planning [[5]](https://app.dosu.dev/7d8810aa-91c6-4074-849b-92fab65de734/documents/09d20768-c6a2-4ac0-9000-4395a9728dc9#L92-L98)
+- **22 MCP tools** — structured tool definitions callable by AI agents via the MCP protocol, covering repo intelligence, PR preflight, scoring, contributor decisions, current-branch analysis, and agent planning [[5]](https://app.dosu.dev/7d8810aa-91c6-4074-849b-92fab65de734/documents/09d20768-c6a2-4ac0-9000-4395a9728dc9#L92-L98)
 - **17 CLI commands** — human-friendly equivalents of the same capabilities, usable directly from a terminal or CI script
 
 It is designed for three audiences:
 
-- **Miners** — use the package to plan work, preflight branches before opening PRs, and prepare public-safe PR packets
+- **Miners** — use the package to plan work, preflight branches before opening PRs, prepare public-safe PR packets, and understand score improvement paths through scenario explanations
 - **Coding agents** — consume deterministic tool schemas to gain structured Gittensor context without requiring source code upload
 - **Maintainers** — access private reviewability context and contributor decision packs through the API
 
@@ -186,7 +186,7 @@
 gittensory-mcp --stdio
 ```
 
-This starts an `McpServer` named `gittensory-local` that communicates via `StdioServerTransport` — reading JSON-RPC messages from stdin and writing responses to stdout [[14]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L151-L154) [[15]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L459). The server registers all 21 MCP tools and remains alive until the client process exits.
+This starts an `McpServer` named `gittensory-local` that communicates via `StdioServerTransport` — reading JSON-RPC messages from stdin and writing responses to stdout [[14]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L151-L154) [[15]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L459). The server registers all 22 MCP tools and remains alive until the client process exits.
 
 The `--stdio` flag is what MCP clients use to spawn and communicate with the server. From the client's perspective, Gittensory appears as a local process that accepts tool calls and returns structured JSON results.
 
@@ -217,7 +217,7 @@
 
 ## What Context It Exposes to Agents
 
-The 21 MCP tools are organized into five capability categories. Together they give agents a complete picture of contribution health — from raw repo signals to scored, ranked next actions — without requiring source code upload.
+The 22 MCP tools are organized into five capability categories. Together they give agents a complete picture of contribution health — from raw repo signals to scored, ranked next actions — without requiring source code upload.
 
 ### Repo Intelligence
 
@@ -235,6 +235,7 @@
 | `gittensory_preflight_pr` | Validates planned PR metadata (title, body, labels, changed files, linked issues, tests) against lane, duplicate, linked-issue, test, and queue signals. Returns a preflight status and a list of findings [[21]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L168-L175) |
 | `gittensory_preflight_local_diff` | Inspects the local git diff metadata and runs the same preflight checks — without uploading source contents. Collects changed files, line counts, and test files locally [[22]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L177-L201) |
 | `gittensory_preview_local_pr_score` | Requests a private scoring preview for the local diff. Accepts optional score overrides (sourceTokenScore, totalTokenScore, sourceLines) and projected contributor state. No source upload [[23]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L212-L219) |
+| `gittensory_explain_score_scenarios` | Explain private score-preview what-if scenarios and gate deltas with ranked cleanup paths. Login and repo scoped; no new computation beyond the preview projection. Returns repoFullName, scoreabilityStatus, effectiveEstimatedScore, headline, scenarios (array of what-if scenario explanations), gateDeltaNarratives (explanations of multiplier stack changes), and recommendedPath (suggested cleanup strategy). Provides human-readable explanations of the six scenario projections (cleanGates, afterPendingMerges, afterApprovedPrsMerge, afterStalePrsClose, linkedIssueFixed, bestReasonableCase) and gate deltas from score preview, helping users understand cleanup priorities and score improvement paths |
 | `gittensory_compare_pr_variants` | Compares private scoring previews across up to 10 metadata variants. Results are sorted by estimated score, making it easy to identify the strongest approach [[24]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L243-L255) |
 
 ### Contributor Decisions
@@ -630,7 +631,7 @@
 
 ### MCP Tools Reference
 
-All 21 tools use Zod-validated input schemas [[53]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L28-L144) and return a structured response containing a human-readable `summary` string and a JSON `data` object.
+All 22 tools use Zod-validated input schemas [[53]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L28-L144) and return a structured response containing a human-readable `summary` string and a JSON `data` object.
 
 #### Category 1: Repo Intelligence
 
@@ -677,6 +678,13 @@
 - **Input:** All fields from `localDiffShape` plus optional `sourceTokenScore`, `totalTokenScore`, `sourceLines`, `credibility`, `projectedCredibility`, PR state fields, `scenarioNotes`, and `branchEligibility` (object with `status`, `source`, `reason`, `checkedAt`, `stale`)
 - **Output:** Score estimate with scenario breakdowns, setup guidance if scorer is missing
 - **Use case:** Get a private score estimate before opening a PR [[23]](https://github.com/JSONbored/gittensory/blob/1e7506f4c5226d0b9b5da8fcd219309d84bacfcd/packages/gittensory-mcp/bin/gittensory-mcp.js#L212-L219)
+
+**`gittensory_explain_score_scenarios`**
+
+- **Purpose:** Explain private score-preview what-if scenarios and gate deltas with ranked cleanup paths
+- **Input:** Same as `gittensory_preview_local_pr_score` (contributorLogin, repoFullName, prNumber, etc.)
+- **Output:** `repoFullName`, `scoreabilityStatus`, `effectiveEstimatedScore`, `headline`, `scenarios` (array of what-if scenario explanations), `gateDeltaNarratives` (explanations of multiplier stack changes), `recommendedPath` (suggested cleanup strategy)
+- **Use case:** Provides human-readable explanations of the six scenario projections (`cleanGates`, `afterPendingMerges`, `afterApprovedPrsMerge`, `afterStalePrsClose`, `linkedIssueFixed`, `bestReasonableCase`) and gate deltas from score preview, helping users understand cleanup priorities and score improvement paths
 
 **`gittensory_compare_pr_variants`**
 

Accept · Edit · Decline


Leave Feedback Ask Dosu about gittensory Add Dosu to your team

@gittensory-orb

gittensory-orb Bot commented Jul 1, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-01 07:00:10 UTC

10 files · 1 AI reviewer · no blockers · readiness 55/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review

Review summary
The change adds a pure score-scenario explanation layer plus API, MCP, OpenAPI, and focused tests around the new surface. The routing and access-control path mirrors the existing score breakdown flow, and the service correctly treats scenario previews as projections over the already-built preview rather than changing scoring behavior. The most notable weakness is that some response schema fields are looser than the TypeScript service contract, but I do not see a reachable correctness break in the provided diff.

Nits — 7 non-blocking
  • nit: src/openapi/schemas.ts:1438 leaves recommendedPath.scenario as an unconstrained string even though the service only returns scenario names; tighten this to the same scenario enum used for cards so generated clients get the real contract.
  • nit: src/mcp/server.ts:877 exposes scenarios, gateDeltaNarratives, and recommendedPath as z.unknown(), which weakens structured MCP validation for the new tool compared with the detailed OpenAPI schema.
  • nit: packages/gittensory-mcp/bin/gittensory-mcp.js:520 estimates source lines by subtracting test file count from changed line count, so local scenario explanations can be noticeably skewed for large test-only changes; consider reusing the existing local preview estimation helper if one exists.
  • src/openapi/schemas.ts:1438 should make recommendedPath.scenario a scenario-name enum, or reuse a shared schema constant to avoid drift with ScenarioCard.name.
  • src/mcp/server.ts:877 should define nested output schemas for the arrays/object rather than z.unknown() so MCP clients catch shape regressions on this new public tool.
  • Readiness score is below the configured threshold — Use the readiness panel as advisory maintainer context; the score does not block this PR.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (no linked issue context).
Validation posture ❌ 5/25 Preflight is holding this PR; address the blocker before review.
Contributor workload ✅ 10/10 Author activity: 99 registered-repo PR(s), 67 merged, 5 issue(s).
Contributor context ✅ Confirmed Gittensor contributor andriypolanski; Gittensor profile; 99 PR(s), 5 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
  • Author: andriypolanski
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 99 PR(s), 5 issue(s).
  • Related work: Titles/paths share 6 meaningful terms. (PR #1984)
  • Related work: Titles/paths share 7 meaningful terms. (PR #1978, PR #1984)
Contributor next steps
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Fix the blocker.
  • Triage stale or unlinked PRs.
  • Refresh registry data or choose a registered active repo.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix these minor issues before I can approve:

The most notable weakness is that some response schema fields are looser than the TypeScript service contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 1.25x multiplier. gittensor Gittensor contributor context

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants