fix(zones): keep a row's zones side by side, and build a shaped node's end from its edges (#584)

docs/flowchart-zones.md

@@ -16,7 +16,7 @@ Additive to [flowchart groups](./flowchart-groups.md):
 
 - `groups[].parent` names the declared group a group sits inside. It mirrors `parent` on `buildStateMachine` states, and an explicit field is the contract the groups plan asked for: nesting is never read out of ids or labels.
 - `groups[].kind` is one of `cloud`, `datacenter`, `region`, `zone`, `vpc`, `subnet`, `cluster` or `namespace`. It says what a zone stands for and sets how it draws. It is a closed set so a model picks a drawn treatment, not free text.
-- `nodes[].shape` is one of `database`, `queue`, `document` or `storage`. It is separate from `kind`, which keeps meaning start, process, decision or end, so a flowchart's start, end and reachability rules hold unchanged: a database a request ends at is `kind: "end", shape: "database"`.
+- `nodes[].shape` is one of `database`, `queue`, `document` or `storage`. It is separate from `kind`, which keeps meaning start, process, decision or end, so a flowchart's start, end and reachability rules hold unchanged: a database a request ends at is `kind: "end", shape: "database"`. Before it validates, a build reads a shaped node's end from its edges: a shaped `process` nothing leaves builds as an `end`, and a shaped `end` that passes data on (a queue its workers read) as a `process` where another end remains, so neither costs the caller a retry.
 - A node sets `group` to its innermost zone. A zone holds nodes directly or through the zones nested in it.
 - Zones nest at most 4 deep (a cloud, a region, a VPC and a subnet), and a flowchart keeps at most 8 groups.
 
@@ -43,13 +43,15 @@ Each zone's label sits at the left of a header strip across its top; nothing the
 - Each zone's members and nested zones are laid out as a flow of their own, innermost first; a nested zone is one block in its parent's flow, the way an outermost zone is one block among the ungrouped nodes. Children stay inside their parents and zones that do not nest never overlap.
 - Edges route around every zone neither end sits in and never cross a zone's label or run along its outline. An edge whose ends share a zone stays inside the innermost zone holding both.
 - Zones keep the diagram's direction at every level: zones a flow passes through one after another follow it, and zones it does not join stand side by side across it.
-- A zoned flow too long for its breadth (2.45:1 for a row at least 1,200 px long or for any column, 4:1 always) wraps as a plain flowchart wraps into rows or columns, never by turning a zone across the flow. Level by level, the top level or one zone's members cut between whole ranks into bands, each read in the stated direction, columns keeping the plain rule's even-split guard. A band hangs under the step that leads into it, as far as the longest band allows, and lines into it turn in from that side. Of the plans that route with no more crossings, export defects or reader findings other than port jogs, the first adding nothing is kept, else the one with the fewest crossings, then findings, then the squarest. The hybrid fixture goes from 4.8:1 to 1.7:1, the AWS one from 2.8:1 to 1.7:1.
+- A zoned flow too long for its breadth (2.45:1 for a row at least 1,200 px long or for any column, 4:1 always) wraps as a plain flowchart wraps into rows or columns, never by turning a zone across the flow. Level by level, the top level or one zone's members cut between whole ranks into bands, each read in the stated direction, columns keeping the plain rule's even-split guard. A band hangs under the step that leads into it, as far as the longest band allows, and lines into it turn in from that side. Of the plans that route with no more crossings, export defects or reader findings other than port jogs, the first adding nothing is kept, else the one with the fewest crossings, then findings, then the squarest. The hybrid fixture goes from 4.8:1 to 1.5:1, the AWS one from 2.8:1 to 1.7:1.
+- In a row, a plan within the target that keeps the top-level zones side by side, wrapping inside one of them, goes before a plan that cuts the top level between two zones. That cut hangs the later zone under the earlier, though the flow passes from one to the next along the row, and the line between them runs down and back against it. The stacked plan stays the first alternative, so a side-by-side plan that routes worse still gives way to it. A column's cut stands the later zone beside the earlier, across a wide export, and columns plan as before. Two of three Sol samples of `hybrid-cloud-zones-v2`, asked for left to right, hung the AWS cloud under the data center (the top level reached the target in two bands, the region in three); the cloud now stands beside it.
 - A plan whose plain routes cross more than the layout it replaces is routed again, each line across as few others as it can, before it is judged, so a crossing the plain router happens to draw does not alone stack sibling zones that read side by side along the flow.
 - A plan that keeps the top level in one band and still crosses one line pair more is judged as the crossing repairs leave it, as the kept plan is drawn, where those leave it crossing, detouring, hooking, running through arrowheads and passing near nodes no more than the layout it replaces, each cleared of the nodes it passes, and no line in it runs more than twice its direct connection past its stubs. Such a plan only keeps sibling zones side by side: it replaces the chosen plan only where that one stacks them and it crosses and finds no more, and it never stacks zones itself. A Sol hybrid sample whose region, wrapped beside the data center, crossed the sync function's lines to the orders table and the invoice PDF once now draws beside it, the table's line entering the table from the side that faces the sync function.
 - A line into a hanging band whose target stands behind another step of its rank, straight across the flow from the source, leaves forward and turns back into the target's forward side, around that step, instead of hooking around the band's margin to reach its near side. This is kept when it routes no worse, hooks or detours no line more, and draws the lines shorter in total. In a round 19 hybrid sample, the line to the orders table drops straight down and the line to the invoice archive brackets around it: four turns fewer and a tenth less line.
 - A zone sits in its rank by its members rather than its outline, so a flow through nested zones runs straight past their header strips.
 - A step reached back through steps outside its zone, as workers fed by a queue the zone's API publishes to, sits beside the step that leaves; outside units in one rank sit under the members they join, a loop back pulling hardest, so the queue sits under its workers.
 - Among zones, no route runs along a node's border.
+- Among zones, a line also keeps the reader's corridor gap (24 px at a 2,000 px fit) from a node it runs alongside but neither leaves nor enters, past a corner's length: the last clearing pass, after every other choice, moves such a run out to the gap where the move brings the line within that clearance of no node and draws nothing worse. A zone whose outline leaves no lane there grows, once every other choice is made, where that routes no worse. Sol and Sonnet hybrid samples ran the ERP's line to the VPN gateway 13 units under the Oracle database's foot, between it and the data center's outline; the data center now grows and the line runs 27 units clear.
 
 ## Excalidraw
 

packages/diagram/agent/src/lib/flowchart/node-kinds.test.ts

@@ -279,6 +279,87 @@ describe("flowchart node kinds", () => {
 		});
 	});
 
+	it("ends a shaped store nothing leaves and steps through a shaped end that passes data on", async () => {
+		// The hybrid sample Sonnet first sent three times: the Oracle database the ERP
+		// stores into, a dead end, marked process.
+		const hybrid = await build({
+			title: "Hybrid setup: on-prem to AWS",
+			layout: { direction: "LR" },
+			groups: [
+				{ id: "dc", label: "On-prem data center", kind: "datacenter" },
+				{ id: "corp", label: "Corporate network", kind: "zone", parent: "dc" },
+				{ id: "aws", label: "AWS cloud", kind: "cloud" },
+				{ id: "reg", label: "eu-west-1", kind: "region", parent: "aws" },
+				{ id: "vpc", label: "Production VPC", kind: "vpc", parent: "reg" },
+				{ id: "priv", label: "Private subnet", kind: "subnet", parent: "vpc" },
+			],
+			nodes: [
+				{ id: "staff", label: "Office staff", kind: "start", group: "corp" },
+				{ id: "erp", label: "Legacy ERP", kind: "process", group: "corp" },
+				{ id: "ora", label: "Oracle database", kind: "process", shape: "database", group: "dc" },
+				{ id: "vpn", label: "Site-to-site VPN gateway", kind: "process", group: "vpc" },
+				{ id: "api", label: "Order API", kind: "process", group: "priv" },
+				{ id: "q", label: "Order queue", kind: "process", shape: "queue", group: "reg" },
+				{ id: "fn", label: "Sync function", kind: "process", group: "reg" },
+				{ id: "tbl", label: "Orders table", kind: "end", shape: "database", group: "reg" },
+				{ id: "arc", label: "Invoice archive", kind: "end", shape: "storage", group: "reg" },
+			],
+			edges: [
+				{ source: "staff", target: "erp", label: "enter orders" },
+				{ source: "erp", target: "ora", label: "stores records" },
+				{ source: "erp", target: "vpn" },
+				{ source: "vpn", target: "api" },
+				{ source: "api", target: "q" },
+				{ source: "q", target: "fn" },
+				{ source: "fn", target: "tbl" },
+				{ source: "fn", target: "arc", label: "saves invoice PDF" },
+			],
+		});
+		if (!hybrid.ok) throw new Error(JSON.stringify(hybrid.issues));
+		expect(hybrid.normalizedSpec.nodes.find((node) => node.id === "ora")?.kind).toBe("end");
+		expect(hybrid.normalizedSpec.nodes.find((node) => node.id === "q")?.kind).toBe("process");
+
+		// The Kubernetes sample Sonnet first sent twice: RabbitMQ, which the workers
+		// read, marked end beside the Cloud SQL end.
+		const cluster = (rabbit: string) => ({
+			title: "GKE cluster setup",
+			nodes: [
+				{ id: "browser", label: "Browser", kind: "start" },
+				{ id: "api", label: "API pods", kind: "process" },
+				{ id: "workers", label: "Worker pods", kind: "process" },
+				{ id: "rmq", label: "RabbitMQ", kind: rabbit, shape: "queue" },
+				{ id: "sql", label: "Cloud SQL", kind: "end", shape: "database" },
+			],
+			edges: [
+				{ source: "browser", target: "api" },
+				{ source: "api", target: "sql", label: "write orders" },
+				{ source: "api", target: "rmq", label: "publish jobs" },
+				{ source: "rmq", target: "workers", label: "consume jobs" },
+				{ source: "workers", target: "sql", label: "write results" },
+			],
+		});
+		const built = await build(cluster("end"));
+		if (!built.ok) throw new Error(JSON.stringify(built.issues));
+		expect(built.normalizedSpec.nodes.find((node) => node.id === "rmq")?.kind).toBe("process");
+
+		// The repair reads only shaped nodes: a plain step nothing leaves still needs
+		// its next edge, and a shaped end that is the flow's only end stays one.
+		expect(
+			kinds({
+				nodes: [
+					{ id: "a", label: "Receive", kind: "start" },
+					{ id: "b", label: "Review", kind: "process" },
+					{ id: "c", label: "Store", kind: "end", shape: "database" },
+				],
+				edges: [
+					{ source: "a", target: "b" },
+					{ source: "a", target: "c" },
+					{ source: "c", target: "b" },
+				],
+			}),
+		).toEqual(["start", "process", "end"]);
+	});
+
 	it("leaves unknown kinds and non-flowchart input alone", () => {
 		const input = { spec: invoice([{ id: "receive", label: "Receive", kind: "subroutine" }]) };
 		expect(resolveFlowchartNodeKinds(input)).toBe(input);

packages/diagram/agent/src/lib/flowchart/node-kinds.ts

@@ -28,6 +28,8 @@ const KIND_SYNONYMS: Readonly<Record<string, FlowchartNodeKind>> = {
 };
 /** Kinds that name a start or an end without saying which: the node's position decides. */
 const TERMINAL_KINDS = new Set(["terminal", "terminator", "terminus"]);
+/** Infrastructure shapes: a node drawn as one is a store or a channel, whatever its kind. */
+const INFRASTRUCTURE_SHAPES = new Set(["database", "document", "queue", "storage"]);
 
 interface Degree {
 	incoming: number;
@@ -41,6 +43,7 @@ interface LooseNode {
 	readonly label?: unknown;
 	readonly kind?: unknown;
 	readonly type?: unknown;
+	readonly shape?: unknown;
 }
 interface LooseEdge {
 	readonly source?: unknown;
@@ -72,7 +75,9 @@ const looseKey = (value: string) =>
  * several outgoing edges, keeps its value so validation asks the caller instead of turning a
  * would-be decision into a step. With another connected end, a non-question process leaf reached
  * by a labeled decision branch is also an end. Ordinary process chains still need their next edge.
- * Edges written { from, to } read as { source, target }.
+ * A connected node drawn as an infrastructure shape is an end when nothing leaves it (a database the
+ * flow stores into) and, with another connected end, a process when something does (a queue its
+ * workers read). Edges written { from, to } read as { source, target }.
  */
 export function resolveFlowchartNodeKinds(input: unknown): unknown {
 	if (!isRecord(input)) return input;
@@ -134,6 +139,13 @@ export function resolveFlowchartNodeKinds(input: unknown): unknown {
 			of(node.id).incoming > 0 &&
 			of(node.id).outgoing === 0,
 	);
+	const shaped = (node: LooseNode) =>
+		typeof node.shape === "string" && INFRASTRUCTURE_SHAPES.has(looseKey(node.shape));
+	// A shaped step nothing leaves is a store the flow ends at, and ends it as surely as an end.
+	const storeEnd = (node: LooseNode, kind: unknown) =>
+		kind === "process" && shaped(node) && of(node.id).incoming > 0 && of(node.id).outgoing === 0;
+	const endRemains =
+		hasConnectedEnd || nodes.some((node, index) => isRecord(node) && storeEnd(node, kinds[index]));
 	const decisions = new Set(
 		nodes.flatMap((node, index) =>
 			isRecord(node) && kinds[index] === "decision" ? [node.id] : [],
@@ -161,6 +173,11 @@ export function resolveFlowchartNodeKinds(input: unknown): unknown {
 		) {
 			kind = "end";
 		}
+		const { incoming, outgoing } = of(node.id);
+		if (storeEnd(node, kind)) kind = "end";
+		// A shaped end that passes data on, as a queue its workers read, is a step.
+		else if (kind === "end" && endRemains && shaped(node) && incoming > 0 && outgoing > 0)
+			kind = "process";
 		if (kind === node.kind) return node;
 		changed = true;
 		const { type: _type, ...rest } = node;

packages/diagram/core/src/fixtures/zones/hybrid-cloud-left-to-right.ts

@@ -0,0 +1,153 @@
+/**
+ * The hybrid setup asked for left to right, whose cloud a top-level wrap hung
+ * under the data center, the ERP's line running down and back left to the VPN
+ * gateway. From a generated eval sample (Sol, the real-voice v2 prompt, #584).
+ */
+export default {
+	id: "hybrid-order-processing-aws-eu-west-1",
+	title: "Hybrid order processing — AWS eu-west-1",
+	nodes: [
+		{
+			id: "staff",
+			label: "Office staff",
+			kind: "start",
+			group: "corp",
+		},
+		{
+			id: "erp",
+			label: "Legacy ERP",
+			kind: "process",
+			group: "corp",
+		},
+		{
+			id: "oracle",
+			label: "Oracle database",
+			kind: "end",
+			icon: {
+				slug: "oracle",
+			},
+			group: "dc",
+			shape: "database",
+		},
+		{
+			id: "vpn",
+			label: "Site-to-site VPN gateway",
+			kind: "process",
+			group: "vpc",
+		},
+		{
+			id: "api",
+			label: "Order API",
+			kind: "process",
+			group: "private",
+		},
+		{
+			id: "queue",
+			label: "Order queue",
+			kind: "process",
+			group: "region",
+			shape: "queue",
+		},
+		{
+			id: "sync",
+			label: "Sync function",
+			kind: "process",
+			group: "region",
+		},
+		{
+			id: "table",
+			label: "Orders table",
+			kind: "end",
+			group: "region",
+			shape: "database",
+		},
+		{
+			id: "archive",
+			label: "Invoice archive",
+			kind: "end",
+			description: "Invoices saved as PDFs",
+			group: "region",
+			shape: "storage",
+		},
+	],
+	edges: [
+		{
+			id: "edge-1",
+			source: "staff",
+			target: "erp",
+			label: "Enter orders",
+		},
+		{
+			id: "edge-2",
+			source: "erp",
+			target: "oracle",
+			label: "Store records",
+		},
+		{
+			id: "edge-3",
+			source: "erp",
+			target: "vpn",
+		},
+		{
+			id: "edge-4",
+			source: "vpn",
+			target: "api",
+		},
+		{
+			id: "edge-5",
+			source: "api",
+			target: "queue",
+		},
+		{
+			id: "edge-6",
+			source: "queue",
+			target: "sync",
+		},
+		{
+			id: "edge-7",
+			source: "sync",
+			target: "table",
+		},
+		{
+			id: "edge-8",
+			source: "sync",
+			target: "archive",
+			label: "Save each invoice as a PDF",
+		},
+	],
+	groups: [
+		{
+			id: "dc",
+			label: "On-prem data center",
+			kind: "datacenter",
+		},
+		{
+			id: "corp",
+			label: "Corporate network",
+			parent: "dc",
+		},
+		{
+			id: "aws",
+			label: "AWS cloud",
+			kind: "cloud",
+		},
+		{
+			id: "region",
+			label: "eu-west-1",
+			parent: "aws",
+			kind: "region",
+		},
+		{
+			id: "vpc",
+			label: "Production VPC",
+			parent: "region",
+			kind: "vpc",
+		},
+		{
+			id: "private",
+			label: "Private subnet",
+			parent: "vpc",
+			kind: "subnet",
+		},
+	],
+} as const;

packages/diagram/core/src/routing-quality.ts

@@ -62,6 +62,11 @@ export function routingCorridorGap(scale: number): number {
 	return CORRIDOR_GAP_PX / scale;
 }
 
+/** Scene-unit length from which side-by-side runs read as a corridor rather than a corner at this scale. */
+export function routingCorridorRun(scale: number): number {
+	return CORRIDOR_RUN_PX / scale;
+}
+
 /** Scene-unit reach from an end within which a kink reads as a port jog at this scale. */
 export function routingJogReach(scale: number): number {
 	return JOG_WINDOW_PX / scale;

packages/diagram/core/src/zone-fixtures.ts

@@ -10,6 +10,7 @@ import hybridCloudBypass from "./fixtures/zones/hybrid-cloud-bypass.js";
 import hybridCloudBypassArchive from "./fixtures/zones/hybrid-cloud-bypass-archive.js";
 import hybridCloudDescribed from "./fixtures/zones/hybrid-cloud-described.js";
 import hybridCloudLabeled from "./fixtures/zones/hybrid-cloud-labeled.js";
+import hybridCloudLeftToRight from "./fixtures/zones/hybrid-cloud-left-to-right.js";
 import hybridCloudStacked from "./fixtures/zones/hybrid-cloud-stacked.js";
 import hybridCloudSync from "./fixtures/zones/hybrid-cloud-sync.js";
 import kindedFlat from "./fixtures/zones/kinded-flat.js";
@@ -92,6 +93,13 @@ export const stackedHybridFlowchart = flowchart(hybridCloudStacked);
  */
 export const corridorHybridFlowchart = flowchart(hybridCloudArchiveCorridor);
 
+/**
+ * The hybrid setup asked for left to right, whose cloud a top-level wrap hung
+ * under the data center, as a generated sample drew it. Its logo needs an
+ * embedded asset to export.
+ */
+export const leftToRightHybridFlowchart = flowchart(hybridCloudLeftToRight);
+
 /** Zones of the thirty-node stress case, each with its parent and kind. */
 const STRESS_ZONES = [
 	{ id: "aws", label: "AWS", kind: "cloud" },

packages/diagram/renderer/src/routing-work.test.ts

@@ -326,6 +326,6 @@ describe("zone routing work", () => {
 		const large = render(zoneStressFixtures);
 		expect(small).toBeGreaterThan(0);
 		expect(small).toBeLessThan(2_300_000);
-		expect(large).toBeLessThan(18_000_000);
+		expect(large).toBeLessThan(24_000_000);
 	});
 });