Skip to content

Commit 05d7545

Browse files
RealDiligentRealDiligent
authored andcommitted
test(scoring): pin the unpinned-fallback and languages-fetch-failed warnings
refreshScoringModelSnapshot pushes two operator-facing warnings that nothing asserted. The unpinned-ref warning was already triggered by the SHA-failure test but never checked, and it is the only signal that scoring is running against a mutable ref. The languages-fetch-failed branch had never executed at all: tests reaching constantsUsable === true always stubbed programming_languages.json to succeed, and tests where it failed had constants.py fail first, short-circuiting earlier. Adds the missing assertion and a constants-succeed/languages-fail case covering the warning and the empty programmingLanguages fallback. Verified both fail when their warning push is removed. Closes #8327
1 parent a20ddd8 commit 05d7545

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

test/unit/scoring.test.ts

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -500,6 +500,31 @@ NOVELTY_BONUS_SCALAR = 3
500500
expect(refreshed.payload.upstreamSourceSha).toBeUndefined();
501501
expect(refreshed.constants.MERGED_PR_BASE_SCORE).toBe(25);
502502
expect(refreshed.sourceKind).toBe("raw-github");
503+
// #8327: the ONLY operator-facing signal that scoring is running unpinned against a mutable ref --
504+
// pin it so a refactor that drops or garbles the string is caught.
505+
expect(refreshed.warnings.some((warning) => /unpinned/i.test(warning))).toBe(true);
506+
});
507+
508+
it("#8327: warns and falls back to empty language weights when ONLY the languages fetch fails", async () => {
509+
// Prior tests that reached constantsUsable === true always stubbed programming_languages.json to
510+
// succeed, and tests where languages failed had constants.py fail first (short-circuiting into the
511+
// earlier !constantsUsable path) -- so this warning-push and its {} fallback never executed.
512+
const env = createTestEnv({ GITHUB_PUBLIC_TOKEN: "token" });
513+
vi.stubGlobal("fetch", async (input: RequestInfo | URL) => {
514+
const url = input.toString();
515+
if (url.includes("constants.py")) return new Response(VALID_CONSTANTS_PY + "MERGED_PR_BASE_SCORE = 25\n");
516+
if (url.includes("programming_languages.json")) return new Response("not found", { status: 404 });
517+
return new Response("not found", { status: 404 });
518+
});
519+
520+
const refreshed = await refreshScoringModelSnapshot(env);
521+
522+
// The constants path still succeeded ...
523+
expect(refreshed.sourceKind).toBe("raw-github");
524+
expect(refreshed.constants.MERGED_PR_BASE_SCORE).toBe(25);
525+
// ... while language weights degrade to {} with an explicit operator-facing warning.
526+
expect(refreshed.programmingLanguages).toEqual({});
527+
expect(refreshed.warnings.some((warning) => /Programming language weights fetch failed/.test(warning))).toBe(true);
503528
});
504529

505530
it("pins the constants fetch to the resolved upstream SHA (immutable) when it can be resolved", async () => {

0 commit comments

Comments
 (0)