Skip to content

Commit 6790312

Browse files
MattJacksonMattJackson
committed
bench: publish run history + regenerated charts/reports
Field-wide artifacts produced after all boxes finished (append-only history + charts.py output).
1 parent 6e25cec commit 6790312

8 files changed

Lines changed: 8 additions & 0 deletions

File tree

‎results/history/agentgateway.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@
99
{"suite":"matrix","measured_at":"2026-07-30T05:27:43Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/agentgateway/agentgateway:v1.4.0"}
1010
{"suite":"matrix","measured_at":"2026-07-30T20:01:28Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/agentgateway/agentgateway:v1.4.0"}
1111
{"suite":"matrix","measured_at":"2026-07-31T10:27:21Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/agentgateway/agentgateway:v1.4.0"}
12+
{"suite":"matrix","measured_at":"2026-08-01T16:29:39Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/agentgateway/agentgateway:v1.4.0"}

‎results/history/aisix.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@
99
{"suite":"matrix","measured_at":"2026-07-30T20:01:36Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"target/release/aisix"}
1010
{"suite":"matrix","measured_at":"2026-07-31T10:27:21Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"target/release/aisix"}
1111
{"suite":"matrix","measured_at":"2026-08-01T02:06:09Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"target/release/aisix"}
12+
{"suite":"matrix","measured_at":"2026-08-01T21:11:29Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"target/release/aisix"}

‎results/history/apisix.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@
99
{"suite":"matrix","measured_at":"2026-07-30T05:27:48Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"apache/apisix:3.17.0-debian"}
1010
{"suite":"matrix","measured_at":"2026-07-30T20:01:32Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"apache/apisix:3.17.0-debian"}
1111
{"suite":"matrix","measured_at":"2026-07-31T10:27:21Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"apache/apisix:3.17.0-debian"}
12+
{"suite":"matrix","measured_at":"2026-08-01T21:11:30Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"apache/apisix:3.17.0-debian"}

‎results/history/bifrost.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,4 @@
1111
{"suite":"matrix","measured_at":"2026-07-30T05:27:52Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"maximhq/bifrost:v1.6.6"}
1212
{"suite":"matrix","measured_at":"2026-07-30T20:01:34Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"maximhq/bifrost:v1.6.6"}
1313
{"suite":"matrix","measured_at":"2026-07-31T10:27:31Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"maximhq/bifrost:v1.6.6"}
14+
{"suite":"matrix","measured_at":"2026-08-01T21:11:32Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"maximhq/bifrost:v1.6.6"}

‎results/history/gomodel.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@
88
{"suite":"matrix","measured_at":"2026-07-30T05:27:56Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"enterpilot/gomodel:0.1.63"}
99
{"suite":"matrix","measured_at":"2026-07-30T20:01:43Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"enterpilot/gomodel:0.1.63"}
1010
{"suite":"matrix","measured_at":"2026-07-31T10:27:31Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"enterpilot/gomodel:0.1.63"}
11+
{"suite":"matrix","measured_at":"2026-08-01T21:11:36Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"enterpilot/gomodel:0.1.63"}

‎results/history/litellm-python.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,4 @@
66
{"suite":"matrix","measured_at":"2026-07-29T13:54:09Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/berriai/litellm:v1.94.0"}
77
{"suite":"matrix","measured_at":"2026-07-30T20:01:52Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/berriai/litellm:v1.94.0"}
88
{"suite":"matrix","measured_at":"2026-07-31T10:27:52Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/berriai/litellm:v1.94.0"}
9+
{"suite":"matrix","measured_at":"2026-08-01T21:11:44Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/berriai/litellm:v1.94.0"}

‎results/history/portkey.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,4 @@
77
{"suite":"matrix","measured_at":"2026-07-30T05:28:17Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"portkeyai/gateway:1.15.2"}
88
{"suite":"matrix","measured_at":"2026-07-30T20:02:01Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"portkeyai/gateway:1.15.2"}
99
{"suite":"matrix","measured_at":"2026-07-31T10:27:51Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"portkeyai/gateway:1.15.2"}
10+
{"suite":"matrix","measured_at":"2026-08-01T21:11:46Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"portkeyai/gateway:1.15.2"}

‎results/history/tensorzero.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@
88
{"suite":"matrix","measured_at":"2026-07-30T05:28:19Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"tensorzero/gateway:2026.6.0"}
99
{"suite":"matrix","measured_at":"2026-07-30T20:02:07Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"tensorzero/gateway:2026.6.0"}
1010
{"suite":"matrix","measured_at":"2026-07-31T10:28:00Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"tensorzero/gateway:2026.6.0"}
11+
{"suite":"matrix","measured_at":"2026-08-01T21:11:45Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"tensorzero/gateway:2026.6.0"}

0 commit comments

Comments
 (0)