Skip to content

Commit 1e1d75b

Browse files
MattJacksonMattJackson
committed
bench: publish run history + regenerated charts/reports
Field-wide artifacts produced after all boxes finished (append-only history + charts.py output).
1 parent f0d1f05 commit 1e1d75b

38 files changed

Lines changed: 73 additions & 61 deletions

‎results/added_latency.png‎

-17.8 KB
Loading

‎results/cost_per_million.png‎

-29.6 KB
Loading

‎results/history/agentgateway.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
{"suite":"matrix","measured_at":"2026-07-28T05:09:16Z","arch":"arm64","hardware":null,"build":"ghcr.io/agentgateway/agentgateway:v1.3.1"}
33
{"suite":"matrix","measured_at":"2026-07-28T07:08:20Z","arch":"arm64","hardware":null,"build":"ghcr.io/agentgateway/agentgateway:v1.3.1"}
44
{"suite":"matrix","measured_at":"2026-07-28T18:44:30Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/agentgateway/agentgateway:v1.3.1"}
5+
{"suite":"matrix","measured_at":"2026-07-28T20:12:39Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/agentgateway/agentgateway:v1.3.1"}

‎results/history/aisix.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
{"suite":"matrix","measured_at":"2026-07-27T04:01:49Z","arch":"arm64","hardware":null,"build":"target/release/aisix"}
22
{"suite":"matrix","measured_at":"2026-07-28T05:09:28Z","arch":"arm64","hardware":null,"build":"target/release/aisix"}
33
{"suite":"matrix","measured_at":"2026-07-28T07:06:47Z","arch":"arm64","hardware":null,"build":"target/release/aisix"}
4+
{"suite":"matrix","measured_at":"2026-07-28T20:12:44Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"target/release/aisix"}

‎results/history/apisix.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
{"suite":"matrix","measured_at":"2026-07-27T04:01:49Z","arch":"arm64","hardware":null,"build":"apache/apisix:3.17.0-debian"}
22
{"suite":"matrix","measured_at":"2026-07-28T05:09:31Z","arch":"arm64","hardware":null,"build":"apache/apisix:3.17.0-debian"}
33
{"suite":"matrix","measured_at":"2026-07-28T07:07:17Z","arch":"arm64","hardware":null,"build":"apache/apisix:3.17.0-debian"}
4+
{"suite":"matrix","measured_at":"2026-07-28T20:12:46Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"apache/apisix:3.17.0-debian"}

‎results/history/bifrost.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
{"suite":"matrix","measured_at":"2026-07-27T23:14:20Z","arch":"arm64","hardware":null,"build":"maximhq/bifrost:v1.6.4"}
33
{"suite":"matrix","measured_at":"2026-07-28T05:09:29Z","arch":"arm64","hardware":null,"build":"maximhq/bifrost:v1.6.4"}
44
{"suite":"matrix","measured_at":"2026-07-28T07:06:48Z","arch":"arm64","hardware":null,"build":"maximhq/bifrost:v1.6.4"}
5+
{"suite":"matrix","measured_at":"2026-07-28T20:12:51Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"maximhq/bifrost:v1.6.4"}

‎results/history/helicone.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
{"suite":"matrix","measured_at":"2026-07-28T00:42:41Z","arch":"arm64","hardware":null,"build":"target/release/ai-gateway"}
33
{"suite":"matrix","measured_at":"2026-07-28T05:09:39Z","arch":"arm64","hardware":null,"build":"target/release/ai-gateway"}
44
{"suite":"matrix","measured_at":"2026-07-28T07:07:49Z","arch":"arm64","hardware":null,"build":"target/release/ai-gateway"}
5+
{"suite":"matrix","measured_at":"2026-07-28T20:13:11Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"target/release/ai-gateway"}

‎results/history/kong.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
{"suite":"matrix","measured_at":"2026-07-27T04:02:08Z","arch":"arm64","hardware":null,"build":"kong:3.8"}
22
{"suite":"matrix","measured_at":"2026-07-28T05:09:42Z","arch":"arm64","hardware":null,"build":"kong:3.8"}
33
{"suite":"matrix","measured_at":"2026-07-28T07:08:05Z","arch":"arm64","hardware":null,"build":"kong:3.8"}
4+
{"suite":"matrix","measured_at":"2026-07-28T20:13:01Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"kong:3.8"}

‎results/history/litellm-python.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
{"suite":"matrix","measured_at":"2026-07-27T04:02:10Z","arch":"arm64","hardware":null,"build":"ghcr.io/berriai/litellm:v1.93.0"}
22
{"suite":"matrix","measured_at":"2026-07-28T05:09:42Z","arch":"arm64","hardware":null,"build":"ghcr.io/berriai/litellm:v1.93.0"}
33
{"suite":"matrix","measured_at":"2026-07-28T07:07:19Z","arch":"arm64","hardware":null,"build":"ghcr.io/berriai/litellm:v1.93.0"}
4+
{"suite":"matrix","measured_at":"2026-07-28T20:13:05Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"ghcr.io/berriai/litellm:v1.93.0"}

‎results/history/litellm-rust.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@
44
{"suite":"matrix","measured_at":"2026-07-28T04:36:15Z","arch":"arm64","hardware":null,"build":"litellm-ai-gateway"}
55
{"suite":"matrix","measured_at":"2026-07-28T05:09:47Z","arch":"arm64","hardware":null,"build":"litellm-ai-gateway"}
66
{"suite":"matrix","measured_at":"2026-07-28T07:07:07Z","arch":"arm64","hardware":null,"build":"litellm-ai-gateway"}
7+
{"suite":"matrix","measured_at":"2026-07-28T20:13:07Z","arch":"arm64","hardware":"AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.","build":"litellm-ai-gateway"}

0 commit comments

Comments
 (0)