Skip to content

Commit c2c6eb3

Browse files
authored
feat(review): let a bot-captured before/after satisfy the screenshot-table gate (#4128)
evaluateScreenshotTableGate now accepts a botCaptureSatisfied input: when the visual-capture pipeline (review.visual.enabled) already rendered a real before/after pair for the PR's current head, the gate is satisfied without a hand-authored body table. The capture result is persisted to a new pull_requests.visual_capture_satisfied_sha column (keyed to head SHA, mirrors approved_head_sha) by maybePublishPrPublicSurface and re-read by the maintenance pass in the same webhook, so no capture is re-run and no return value needs threading through every caller. Also resolves the dead ScreenshotTableGateAction surface: request_changes and comment were fully typed/validated but processors.ts only ever branched on "close", so setting either silently did nothing. Both are removed; "close" is now the only valid action, and a legacy config value normalizes to it with a warning like any other invalid input.
1 parent 5bfa6cb commit c2c6eb3

18 files changed

Lines changed: 509 additions & 43 deletions

.gittensory.yml.example

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -822,13 +822,14 @@ settings:
822822
# minConfidence: 0.85 # Number 0-1. Minimum AI-verifier confidence to treat a candidate as a real match.
823823

824824
# Before/after screenshot-table gate (#2006): deterministic check that a visual/frontend PR's body has a
825-
# before/after image table. Scoped to whenLabels OR whenPaths (either matches); both empty = every PR. Off
826-
# by default.
825+
# before/after image table -- OR (#4110) that the bot's own visual-capture pipeline (review.visual.enabled)
826+
# already produced a real before/after render for this PR's head, which satisfies the gate on its own.
827+
# Scoped to whenLabels OR whenPaths (either matches); both empty = every PR. Off by default.
827828
# screenshotTableGate:
828829
# enabled: false # Default: false.
829830
# whenLabels: [frontend, visual] # Default: [] (no label scoping).
830831
# whenPaths: ["apps/ui/**", "src/**/*.tsx"] # Default: [] (no path scoping).
831-
# action: close # close | request_changes | comment. Default: close.
832+
# action: close # close is the only supported value. Default: close.
832833
# message: "Custom close reason..." # Default: null (built-in message).
833834

834835
# Maintainer AI review tuning (`.gittensory.yml` top-level `review:` block). These knobs shape the advisory AI

apps/gittensory-ui/public/openapi.json

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9265,9 +9265,7 @@
92659265
"action": {
92669266
"type": "string",
92679267
"enum": [
9268-
"close",
9269-
"request_changes",
9270-
"comment"
9268+
"close"
92719269
]
92729270
},
92739271
"message": {

config/examples/gittensory.full.yml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -835,13 +835,14 @@ settings:
835835
# minConfidence: 0.85 # Number 0-1. Minimum AI-verifier confidence to treat a candidate as a real match.
836836

837837
# Before/after screenshot-table gate (#2006): deterministic check that a visual/frontend PR's body has a
838-
# before/after image table. Scoped to whenLabels OR whenPaths (either matches); both empty = every PR. Off
839-
# by default.
838+
# before/after image table -- OR (#4110) that the bot's own visual-capture pipeline (review.visual.enabled)
839+
# already produced a real before/after render for this PR's head, which satisfies the gate on its own.
840+
# Scoped to whenLabels OR whenPaths (either matches); both empty = every PR. Off by default.
840841
# screenshotTableGate:
841842
# enabled: false # Default: false.
842843
# whenLabels: [frontend, visual] # Default: [] (no label scoping).
843844
# whenPaths: ["apps/ui/**", "src/**/*.tsx"] # Default: [] (no path scoping).
844-
# action: close # close | request_changes | comment. Default: close.
845+
# action: close # close is the only supported value. Default: close.
845846
# message: "Custom close reason..." # Default: null (built-in message).
846847

847848
# Maintainer AI review tuning (`.gittensory.yml` top-level `review:` block). These knobs shape the advisory AI
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
-- Visual-capture gate satisfaction (#4110, visual-capture convergence epic #3607). The bot's before/after
2+
-- capture pipeline (review.visual.enabled, #4093) can now satisfy the deterministic screenshotTableGate
3+
-- (#2006) exactly like a hand-authored before/after table -- but the capture is computed and persisted by the
4+
-- public-surface publish pass (maybePublishPrPublicSurface), which runs BEFORE the maintenance/gate pass
5+
-- (maybeRunAgentMaintenance) re-reads this same PR row. Persisting the marker lets the maintenance pass see
6+
-- "did the bot already prove this PR visually?" without re-running the capture or threading a new return value
7+
-- through every caller of either function.
8+
--
9+
-- visual_capture_satisfied_sha is the head SHA at which the capture pipeline last produced a REAL before+after
10+
-- render pair (not a placeholder/failed/pending shot) -- scoped to head SHA (mirrors approved_head_sha, 0053 /
11+
-- last_published_surface_sha, 0080: a new commit re-arms the requirement until capture succeeds again for the
12+
-- new head).
13+
ALTER TABLE pull_requests ADD COLUMN visual_capture_satisfied_sha TEXT;

packages/gittensory-engine/src/review/screenshot-table-gate.ts

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ export const DEFAULT_SCREENSHOT_TABLE_GATE: ScreenshotTableGateConfig = {
2828
action: "close",
2929
};
3030

31-
const VALID_ACTIONS: readonly ScreenshotTableGateAction[] = ["close", "request_changes", "comment"];
31+
const VALID_ACTIONS: readonly ScreenshotTableGateAction[] = ["close"];
3232

3333
export function isScreenshotTableGateAction(value: unknown): value is ScreenshotTableGateAction {
3434
return typeof value === "string" && (VALID_ACTIONS as readonly string[]).includes(value);
@@ -72,7 +72,7 @@ export function normalizeScreenshotTableGateConfig(input: unknown, warnings: str
7272
const action = isScreenshotTableGateAction(record.action)
7373
? record.action
7474
: (() => {
75-
if (record.action !== undefined) warnings.push(`settings.requireScreenshotTable.action must be one of close, request_changes, comment; using the default "close".`);
75+
if (record.action !== undefined) warnings.push(`settings.requireScreenshotTable.action must be "close" (the only supported value; #4110 removed request_changes/comment as dead config surface); using the default "close".`);
7676
return DEFAULT_SCREENSHOT_TABLE_GATE.action;
7777
})();
7878
const message = typeof record.message === "string" && record.message.trim().length > 0 ? record.message.trim() : undefined;
@@ -184,16 +184,25 @@ const NO_VIOLATION: ScreenshotTableGateResult = { violated: false, reason: null
184184

185185
/** PURE evaluator. Off (`enabled: false`) or out-of-scope (no configured label/path match) ⇒ no violation. In
186186
* scope AND (no image-bearing table in the body OR an image pasted outside a table OR a committed image file
187-
* under a scoped path) ⇒ violated, with the configured (or default) templated message as the reason. */
187+
* under a scoped path), UNLESS `botCaptureSatisfied` ⇒ violated, with the configured (or default) templated
188+
* message as the reason. */
188189
export function evaluateScreenshotTableGate(input: {
189190
config: ScreenshotTableGateConfig;
190191
prBody: string | null | undefined;
191192
prLabels: string[];
192193
changedFiles: string[];
194+
/** #4110: true when the bot's own before/after capture pipeline (review.visual.enabled) already produced a
195+
* REAL before+after render pair for this PR's current head — evidence equivalent to a hand-authored table.
196+
* A successful automated capture satisfies the gate on its own, ahead of (and regardless of) the body-table
197+
* anti-gaming checks below — those exist to stop a contributor from FAKING compliance without the bot's
198+
* help, which doesn't apply once the bot has already proven the change visually. Absent/false ⇒
199+
* byte-identical to pre-#4110 behavior (body-table evidence only). */
200+
botCaptureSatisfied?: boolean | undefined;
193201
}): ScreenshotTableGateResult {
194202
const { config } = input;
195203
if (!config.enabled) return NO_VIOLATION;
196204
if (!isScreenshotTableGateInScope(config, input.prLabels, input.changedFiles)) return NO_VIOLATION;
205+
if (input.botCaptureSatisfied === true) return NO_VIOLATION;
197206
const hasTable = hasImageBearingMarkdownTable(input.prBody);
198207
const outsideTable = hasImageOutsideTable(input.prBody);
199208
const committedImage = hasCommittedImageFile(input.changedFiles, config.whenPaths);

packages/gittensory-engine/src/types/manifest-deps-types.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,9 @@ export type CombineStrategy = "single" | "consensus" | "synthesis";
1818

1919
export type OnMerge = "either" | "both";
2020

21-
export type ScreenshotTableGateAction = "close" | "request_changes" | "comment";
21+
// #4110: `request_changes`/`comment` were REMOVED (see src/types.ts's mirror of this type for why) -- `"close"`
22+
// is the only value this gate has ever enforced.
23+
export type ScreenshotTableGateAction = "close";
2224

2325
export type ScreenshotTableGateConfig = {
2426
enabled: boolean;

src/db/repositories.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3657,6 +3657,19 @@ export async function markPullRequestSurfacePublished(env: Env, fullName: string
36573657
.where(and(eq(pullRequests.repoFullName, fullName), eq(pullRequests.number, number), eq(pullRequests.headSha, headSha)));
36583658
}
36593659

3660+
/** Visual-capture gate satisfaction (#4110): record the head SHA at which the bot's before/after capture
3661+
* pipeline just produced a REAL before+after render pair for this PR (see `hasSuccessfulBotCapture`,
3662+
* `review/visual/capture.ts`). The screenshotTableGate evaluator treats `visualCaptureSatisfiedSha ===
3663+
* headSha` as evidence equivalent to a hand-authored table. Scoped to headSha (mirrors markPullRequestApproved)
3664+
* so a later commit re-arms the requirement until capture succeeds again for the new head. */
3665+
export async function markPullRequestVisualCaptureSatisfied(env: Env, fullName: string, number: number, headSha: string): Promise<void> {
3666+
const db = getDb(env.DB);
3667+
await db
3668+
.update(pullRequests)
3669+
.set({ visualCaptureSatisfiedSha: headSha, updatedAt: nowIso() })
3670+
.where(and(eq(pullRequests.repoFullName, fullName), eq(pullRequests.number, number), eq(pullRequests.headSha, headSha)));
3671+
}
3672+
36603673
/** Sweep convergence: stamp the timestamp the scheduled re-gate sweep just recomputed this PR. A plain D1 UPDATE
36613674
* — NOT routed through the agent-action-executor chokepoint (#1258) — so it advances even when GitHub writes are
36623675
* suppressed (dry-run / paused). selectRegateCandidates orders the sweep by last_regated_at, so a just-regated PR
@@ -5799,6 +5812,7 @@ function toPullRequestRecordFromRow(row: typeof pullRequests.$inferSelect): Pull
57995812
lastPublishedSurfaceSha: row.lastPublishedSurfaceSha,
58005813
linkedIssueHardRuleViolatedAt: row.linkedIssueHardRuleViolatedAt,
58015814
linkedIssueHardRuleViolationReason: row.linkedIssueHardRuleViolationReason,
5815+
visualCaptureSatisfiedSha: row.visualCaptureSatisfiedSha,
58025816
};
58035817
}
58045818

src/db/schema.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -477,6 +477,14 @@ export const pullRequests = sqliteTable(
477477
// pairing with merge_blocked_sha) -- so a later close can still cite the concrete rule even if the live
478478
// re-parse can no longer reproduce it (the issue was unlinked or its state changed).
479479
linkedIssueHardRuleViolationReason: text("linked_issue_hard_rule_violation_reason"),
480+
// Visual-capture gate satisfaction (#4110): the head SHA at which the bot's before/after capture pipeline
481+
// (review.visual.enabled) last produced a REAL before+after render pair (not a placeholder/failed/pending
482+
// shot) for this PR. Lets the deterministic screenshotTableGate treat a successful automated capture as
483+
// equivalent evidence to a hand-authored before/after table. Keyed to head SHA (mirrors approved_head_sha /
484+
// last_published_surface_sha) -- a new commit re-arms the requirement until capture succeeds again for the
485+
// new head. gittensory-computed (publish-written), omitted from the GitHub-sync SET clause so a later sync
486+
// cannot clobber it.
487+
visualCaptureSatisfiedSha: text("visual_capture_satisfied_sha"),
480488
createdAt: text("created_at").notNull().$defaultFn(() => nowIso()),
481489
updatedAt: text("updated_at").notNull().$defaultFn(() => nowIso()),
482490
},

src/openapi/schemas.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -776,7 +776,7 @@ export const RepositorySettingsSchema = z
776776
enabled: z.boolean(),
777777
whenLabels: z.array(z.string()),
778778
whenPaths: z.array(z.string()),
779-
action: z.enum(["close", "request_changes", "comment"]),
779+
action: z.enum(["close"]),
780780
message: z.string().optional(),
781781
})
782782
.optional(),

src/queue/processors.ts

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ import {
5656
markPullRequestsRegated,
5757
markPullRequestReviewsInvalidated,
5858
markPullRequestSurfacePublished,
59+
markPullRequestVisualCaptureSatisfied,
5960
getLatestRegatedAt,
6061
claimRegateFanoutSlot,
6162
recordAgentCommandFeedback,
@@ -358,7 +359,7 @@ import { randomUUID } from "node:crypto";
358359
import { isRetryableJobError, RetryableJobError } from "./retryable";
359360
import { screenshotsAllowed } from "../review/visual-wire";
360361
import { isVisualPath } from "../review/visual/paths";
361-
import { buildCapture, type CaptureRoute } from "../review/visual/capture";
362+
import { buildCapture, hasSuccessfulBotCapture, type CaptureRoute } from "../review/visual/capture";
362363
import { incr } from "../selfhost/metrics";
363364
import {
364365
renderReviewingPlaceholder,
@@ -2778,17 +2779,22 @@ async function runAgentMaintenancePlanAndExecute(
27782779
);
27792780

27802781
// Screenshot-table gate (#2006): a DETERMINISTIC check (no AI) that an in-scope (label/path-matched)
2781-
// contributor visual/frontend PR's body contains a before/after screenshot table. Off by default
2782-
// (settings.screenshotTableGate.enabled === false), so the pure evaluator below is effectively free for the
2783-
// common case. Only "close" is wired as an enforcement action here (the other configured actions stay
2784-
// advisory, matching the issue's phased rollout) -- the ternary below is the ONLY place that reads `.action`.
2782+
// contributor visual/frontend PR's body contains a before/after screenshot table -- OR (#4110) that the
2783+
// bot's own visual-capture pipeline already produced a real before/after render for this exact head
2784+
// (markPullRequestVisualCaptureSatisfied, written earlier in this same webhook by maybePublishPrPublicSurface
2785+
// -- see that function's beforeAfter block -- and re-read here on `pr`, which this caller already re-fetched
2786+
// fresh from the DB). Off by default (settings.screenshotTableGate.enabled === false), so the pure evaluator
2787+
// below is effectively free for the common case. "close" is the only enforcement action this gate has (#4110
2788+
// removed the dead request_changes/comment surface) -- the check below is the ONLY place that reads `.action`.
27852789
/* v8 ignore next -- defensive: resolveRepositorySettings always populates screenshotTableGate (getRepositorySettings's DB defaults), so this fallback is unreachable in practice. */
27862790
const screenshotTableGateConfig = settings.screenshotTableGate ?? DEFAULT_SCREENSHOT_TABLE_GATE;
2791+
const botCaptureSatisfied = Boolean(pr.headSha) && pr.visualCaptureSatisfiedSha === pr.headSha;
27872792
const screenshotTableGateResult = evaluateScreenshotTableGate({
27882793
config: screenshotTableGateConfig,
27892794
prBody: pr.body,
27902795
prLabels: pr.labels,
27912796
changedFiles: changedPaths,
2797+
botCaptureSatisfied,
27922798
});
27932799
const screenshotTableMatch =
27942800
screenshotTableGateResult.violated && screenshotTableGateConfig.action === "close"
@@ -10260,6 +10266,24 @@ async function maybePublishPrPublicSurface(
1026010266
? { routes: [], previewPending: false }
1026110267
: await buildCapture(env, token, captureTarget, visualFiles, githubRateLimitAdmissionKeyForInstallation(installationId), reviewVisualConfig);
1026210268
beforeAfter = capture.routes;
10269+
// Screenshot-table gate satisfaction (#4110): a successful capture (a real before+after render pair
10270+
// on at least one route) is evidence equivalent to a hand-authored before/after table -- persist the
10271+
// head SHA it was proven at so the LATER maintenance pass (runAgentMaintenancePlanAndExecute, which
10272+
// re-reads this PR row fresh) can see it without re-running the capture or threading a new return
10273+
// value through every caller of this function. Best-effort: a write failure here just means the gate
10274+
// falls back to requiring a body table, never blocks the rest of the review.
10275+
if (pr.headSha && hasSuccessfulBotCapture(beforeAfter)) {
10276+
await markPullRequestVisualCaptureSatisfied(env, repoFullName, pr.number, pr.headSha).catch((error) => {
10277+
console.log(
10278+
JSON.stringify({
10279+
event: "visual_capture_satisfied_mark_failed",
10280+
repoFullName,
10281+
pull: pr.number,
10282+
message: errorMessage(error).slice(0, 200),
10283+
}),
10284+
);
10285+
});
10286+
}
1026310287
// Visual self-poll: the FIRST capture returns a "loading" placeholder for the AFTER shot when the
1026410288
// preview deploy isn't live yet (capture.previewPending). Schedule a delayed re-review to re-capture
1026510289
// the now-ready shot — bounded by `attempt` so a never-resolving preview can't loop (the deployment_status

0 commit comments

Comments
 (0)