Improve checkout raster commerce corroboration

Author: andresdefi

docs/AI_DESIGN_SYSTEM_ROADMAP.md

@@ -119,7 +119,7 @@ Current status:
 - local semantic cue families now also separate editor/canvas, catalog/store, profile/community, activity/feed, document/review, map/navigation, media/player, capture/scan, schedule/calendar, commerce/checkout, secure-access, support/help, and rewards/loyalty screenshots from those broader buckets, reducing more note/filename-driven non-OCR false positives without bundling built-in model dependencies
 - screenshot analysis now persists semantic-family labels plus confidence for those local cue families, and stronger raster-only ambiguity guards reject weak special-family matches when generic settings/control evidence dominates
 - screenshot analysis now also persists semantic-family rationale, competing family candidates, and review-needed flags so weak local matches are easier to inspect before they steer planning
-- raster-family structural scoring now also helps more distinctive non-OCR profile/document/editor/catalog/map/media/capture layouts plus support/help-style list flows resolve into the right semantic family without OCR text, and stronger settings/paywall ambiguity guards now block more weak profile/reward/commerce family overreads from filenames or terse notes alone
+- raster-family structural scoring now also helps more distinctive non-OCR profile/document/editor/catalog/map/media/capture layouts plus support/help-style list flows resolve into the right semantic family without OCR text, surfaces checkout-style summary/price-rail rows as stronger reviewed commerce contenders, and stronger settings/paywall ambiguity guards still block more weak profile/reward/commerce family overreads from filenames or terse notes alone
 - preview sessions now support manual semantic-family override/reset with session persistence, surfaced rationale/ambiguity diagnostics, and bulk family-review actions for flagged screenshots, so borderline local screenshot classification can be corrected faster during review without adding a built-in model dependency
 - reviewed semantic-family state can now also feed forward into deterministic replanning/materialization through `appframe_plan_variant_set` reviewed-analysis input and `appframe_rebuild_autopilot_session_from_review`, so saved review metadata is no longer preview-only
 - variant plans now emit explicit per-concept frame strategies plus per-screen/per-frame crop plans, including text-occupied-region avoidance and focal-point anchoring

docs/NEXT_STEPS.md

@@ -61,7 +61,7 @@ AppFrame now has an initial autopilot pipeline implemented:
 - screenshot analysis now also broadens deterministic non-OCR understanding for workflow/action and discovery/browse screens, and local ambiguity guards reduce some raster-only false positives where wide panels previously collapsed into settings or reporting
 - screenshot analysis now also uses stronger local cue families for editor/canvas, catalog/store, profile/community, activity/feed, document/review, map/navigation, media/player, capture/scan, schedule/calendar, commerce/checkout, secure-access, support/help, and rewards/loyalty screenshots, widening deterministic understanding and cutting more raster-only settings/reporting/editor/media/paywall false positives without bundling built-in model dependencies
 - screenshot analysis now persists explicit semantic-family metadata plus confidence for those local cue families, and stricter raster-only ambiguity guards keep generic settings/control surfaces from over-triggering special family treatments off weak filename/note cues alone
-- screenshot analysis now also uses broader raster-family structural scoring for more distinctive non-OCR profile/document/editor/catalog/map/media/capture-like layouts, now gives support/help-style list flows stronger raster corroboration, and keeps checkout/paywall ambiguity more conservative so weak profile/reward/commerce hints do not override generic control or premium CTA structure as often
+- screenshot analysis now also uses broader raster-family structural scoring for more distinctive non-OCR profile/document/editor/catalog/map/media/capture-like layouts, now gives support/help-style list flows stronger raster corroboration, surfaces checkout-style summary/price-rail rows as stronger reviewed commerce contenders, and keeps checkout/paywall ambiguity conservative so weak profile/reward/commerce hints do not override generic control or premium CTA structure as often
 - dynamic individual planning/materialization now reacts more explicitly to onboarding, paywall, settings, chat, and reporting-style screens with role-aware composition/background strategies instead of only generic frameStrategy/cropPlan behavior
 - dynamic individual planning/materialization now also reacts more explicitly to workflow and discovery screens with dedicated composition/background treatments instead of folding them back into generic proof-grid behavior
 - dynamic planning/materialization now also reacts more explicitly to editor/profile/catalog/activity/document/map/media/capture/schedule/commerce/security/support/reward-style screens with dedicated individual backgrounds, copy guidance, family-aware panoramic pacing, and richer panoramic tool-ribbon / profile-spotlight / activity-wave / folio-stack / browse-strip / route-arc / playback-marquee / capture-focus / timeline-band / checkout-lane / trust-shield / support-beacon / reward-ribbon treatments
@@ -262,7 +262,7 @@ Recommended order:
   - [x] data-heavy dashboard/reporting
   - [x] workflow/action
   - [x] discovery/browse/template-library
-  Status: OCR/layout semantics now improve these cases when text enrichment is available, raster/non-OCR deterministic logic now covers the same families plus workflow/discovery and local editor/profile/catalog/activity/document/map/media/capture/schedule/commerce/security/support/reward cue families, raster-family scoring now helps more structurally distinctive non-OCR profile/document/editor/catalog/map/media/capture screens plus support/help-style list flows resolve into the right semantic family even with generic filenames, and ambiguity guards now cut more settings/reporting/profile/reward/commerce/media/paywall false positives; broader scene-graph understanding is still incomplete.
+  Status: OCR/layout semantics now improve these cases when text enrichment is available, raster/non-OCR deterministic logic now covers the same families plus workflow/discovery and local editor/profile/catalog/activity/document/map/media/capture/schedule/commerce/security/support/reward cue families, raster-family scoring now helps more structurally distinctive non-OCR profile/document/editor/catalog/map/media/capture screens plus support/help-style list flows resolve into the right semantic family even with generic filenames, checkout-style summary/price-rail rows now surface stronger reviewed commerce contenders, and ambiguity guards still cut more settings/reporting/profile/reward/commerce/media/paywall false positives; broader scene-graph understanding is still incomplete.
 - [x] Add raster-only occupied-region heuristics when OCR/text enrichment is absent.
   Status: screenshot analysis now infers top/bottom/left/right/center occupancy from local PNG structure, and those regions feed crop avoidance plus copy/planning guidance even without OCR sidecars.
 - [x] Add screenshot ordering inference from filenames, timestamps, and roles.

packages/mcp-server/src/tools/design-planning.test.ts

@@ -537,7 +537,6 @@ describe('design planning helpers', () => {
       { path: detailPath, note: 'Order detail and delivery status' },
     ]);
     const settingsAnalysis = analysis.find((entry) => entry.path === settingsPath);
-
     expect(settingsAnalysis?.role).toBe('settings');
     expect(settingsAnalysis?.semanticFlavor).toBeUndefined();
 
@@ -683,6 +682,37 @@ describe('design planning helpers', () => {
     expect(supportAnalysis?.semanticFlavorReason?.some((line) => line.includes('raster structure'))).toBe(true);
   });
 
+  it('surfaces commerce as a reviewed contender for checkout-like raster screens without OCR or descriptive filenames', async () => {
+    const commercePath = await makePngFile('screen-7.png', 120, 200, (x, y) => {
+      if (y < 10) return [255, 251, 235, 255];
+      if (x > 18 && x < 102 && y > 20 && y < 70) return [254, 215, 170, 255];
+      if (
+        ((x > 18 && x < 90) && (y > 86 && y < 102))
+        || ((x > 18 && x < 90) && (y > 110 && y < 126))
+      ) {
+        return [148, 163, 184, 255];
+      }
+      if (
+        ((x > 96 && x < 110) && (y > 86 && y < 102))
+        || ((x > 96 && x < 110) && (y > 110 && y < 126))
+      ) {
+        return [30, 64, 175, 255];
+      }
+      if (x > 28 && x < 92 && y > 150 && y < 172) return [37, 99, 235, 255];
+      return [255, 247, 237, 255];
+    });
+
+    const analysis = await analyzeScreenshotSet([
+      { path: commercePath },
+    ]);
+
+    const commerceAnalysis = analysis.find((entry) => entry.path === commercePath);
+    expect(commerceAnalysis?.role).not.toBe('paywall');
+    expect(commerceAnalysis?.semanticFlavor).not.toBe('support');
+    expect(commerceAnalysis?.semanticFlavorNeedsReview).toBe(true);
+    expect(commerceAnalysis?.semanticFlavorAlternatives?.some((entry) => entry.flavor === 'commerce')).toBe(true);
+  });
+
   it('rejects weak profile and reward family matches when generic settings structure dominates', async () => {
     const profileSettingsPath = await makePngFile('screen-4.png', 120, 200, (x, y) => {
       if (y < 28) return [248, 250, 252, 255];

packages/mcp-server/src/tools/design-planning.ts

@@ -780,6 +780,7 @@ function inferSemanticFlavorDetails(args: {
   const profileScore = (signals.profileSignalCount * 2)
     + (args.role === 'detail' || args.role === 'communication' || args.role === 'discovery' ? 1 : 0)
     + rasterFlavorBonus(profileRasterScore)
+    + (profileRasterScore >= 4 ? 1 : 0)
     + (profileRasterScore > activityRasterScore ? 1 : 0)
     - (signals.settingsConflictCount >= 2 ? 3 : signals.settingsConflictCount);
   const editorScore = (signals.editorSignalCount * 2)
@@ -796,6 +797,7 @@ function inferSemanticFlavorDetails(args: {
     ? (signals.documentSignalCount * 2)
       + (args.role === 'detail' ? 2 : args.role === 'workflow' ? 1 : 0)
       + rasterFlavorBonus(documentRasterScore)
+      + (documentRasterScore >= 5 ? 1 : 0)
       + (signals.scheduleSignalCount >= 1 ? 1 : 0)
       - Math.min(2, signals.reportingConflictCount)
       - Math.min(1, signals.editorSignalCount)
@@ -804,6 +806,7 @@ function inferSemanticFlavorDetails(args: {
       - Math.min(1, signals.paywallConflictCount)
     : ((args.role === 'detail' ? 2 : args.role === 'workflow' ? 1 : 0)
       + rasterFlavorBonus(documentRasterScore)
+      + (documentRasterScore >= 5 ? 1 : 0)
       - (editorRasterScore > documentRasterScore ? 2 : 0));
   const mapScore = (signals.mapSignalCount * 2)
     + (args.role === 'discovery' || args.role === 'home' ? 2 : args.role === 'detail' ? 1 : 0)
@@ -819,7 +822,9 @@ function inferSemanticFlavorDetails(args: {
     + (args.role === 'detail' || args.role === 'workflow' ? 2 : args.role === 'discovery' || args.role === 'home' ? 1 : 0)
     + rasterFlavorBonus(commerceRasterScore)
     + (commerceRasterScore >= 6 ? 2 : commerceRasterScore >= 5 ? 1 : 0)
+    + (commerceRasterScore >= 6 && (args.role === 'detail' || args.role === 'workflow') ? 2 : commerceRasterScore >= 6 ? 1 : 0)
     + (commerceRasterScore > scheduleRasterScore ? 1 : 0)
+    + (commerceRasterScore > supportRasterScore ? 2 : 0)
     + (signals.catalogSignalCount >= 1 ? 1 : 0)
     - (signals.paywallConflictCount > signals.commerceSignalCount && commerceRasterScore < 6 ? 2 : 0)
     - Math.min(1, signals.settingsConflictCount);
@@ -2634,6 +2639,7 @@ function inferRasterSemanticSignals(args: {
   let feedRows = 0;
   let upperHeroPanelRows = 0;
   let agendaRows = 0;
+  let priceRailRows = 0;
   let previousBubbleSide: 'left' | 'right' | null = null;
 
   for (let row = 0; row < args.blockRows; row += 1) {
@@ -2731,6 +2737,36 @@ function inferRasterSemanticSignals(args: {
     ) {
       agendaRows += 1;
     }
+
+    if (
+      segments.length === 3
+      && rowCenter >= 0.22
+      && rowCenter <= 0.86
+      && segments[0]!.start <= 1
+      && segments[0]!.width >= 1
+      && segments[0]!.width <= 2
+      && segments[1]!.width >= 3
+      && segments[1]!.width <= 7
+      && segments[2]!.end >= args.blockColumns - 2
+      && segments[2]!.width >= 1
+      && segments[2]!.width <= 2
+    ) {
+      priceRailRows += 1;
+    }
+
+    if (
+      segments.length === 2
+      && rowCenter >= 0.22
+      && rowCenter <= 0.86
+      && segments[0]!.start <= 2
+      && segments[0]!.width >= 5
+      && segments[0]!.end <= args.blockColumns - 3
+      && segments[1]!.end >= args.blockColumns - 2
+      && segments[1]!.width >= 1
+      && segments[1]!.width <= 2
+    ) {
+      priceRailRows += 1;
+    }
   }
 
   const occupiedRegions = mergeOccupiedRegions(
@@ -2869,10 +2905,15 @@ function inferRasterSemanticSignals(args: {
     + (splitRows >= 2 ? 1 : 0)
   );
   const commerceFlavorScore = (
-    (agendaRows >= 3 ? 2 : agendaRows >= 2 ? 1 : 0)
+    (agendaRows >= 4 ? 3 : agendaRows >= 3 ? 2 : agendaRows >= 2 ? 1 : 0)
+    + (priceRailRows >= 3 ? 3 : priceRailRows >= 2 ? 2 : priceRailRows >= 1 ? 1 : 0)
     + (lowerCtaRows >= 1 ? 2 : 0)
     + (upperHeroPanelRows >= 2 ? 1 : 0)
     + (topInsetBarRows >= 1 ? 1 : 0)
+    + (agendaRows >= 2 && centeredPanelRows >= 3 ? 1 : 0)
+    + (agendaRows >= 2 && centeredPanelRows >= 3 && lowerCtaRows >= 1 ? 2 : 0)
+    + (agendaRows >= 2 && fullWidthRows >= 2 ? 1 : 0)
+    - (agendaRows <= 1 && args.averageLuminance <= 148 ? 1 : 0)
   );
   const securityFlavorScore = (
     (centeredPanelRows >= 4 ? 2 : centeredPanelRows >= 3 ? 1 : 0)
@@ -2891,6 +2932,8 @@ function inferRasterSemanticSignals(args: {
     + (feedRows <= 2 ? 1 : 0)
     + (cardGridRows <= 1 ? 1 : 0)
     + (splitRows >= 2 ? 1 : 0)
+    - (priceRailRows >= 1 ? 2 : 0)
+    - (agendaRows >= 2 && centeredPanelRows >= 3 && lowerCtaRows >= 1 ? 2 : 0)
   );
   const rewardFlavorScore = (
     (upperHeroPanelRows >= 3 ? 2 : upperHeroPanelRows >= 2 ? 1 : 0)