Skip to content

Commit 0a64bb7

Browse files
MattJacksonMattJackson
committed
results: full field run - all gateways x all suites (perf, memory, stream, xlate, governed, matrix)
1 parent fbaeaac commit 0a64bb7

2 files changed

Lines changed: 18 additions & 15 deletions

File tree

‎results/reports/all/README.md‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,23 +19,26 @@ Every number below is regenerated from the raw `results/*.json` — re-run `run-
1919
| [One-API](https://github.com/songquanpeng/one-api) | 34,637 µs | 0 | 0 | 84 MiB | 20124 MiB | `justsong/one-api:v0.6.10 (@sha256:e667221a2e19` |
2020
| [TensorZero](https://github.com/tensorzero/tensorzero) | 40,946 µs | 4,179 | 12,141 | 49 MiB | 700 MiB | `tensorzero/gateway:2026.6.0 (@sha256:c939db4f2` |
2121
| [Arch](https://github.com/katanemo/archgw) | 255,873 µs | 0 | 0 | 471 MiB | 1325 MiB | `katanemo/archgw:0.3.22 (archgw CLI)` |
22+
| [Busbar (main)](https://github.com/GetBusbar/busbar) | ⏳ *pending* | — | — | — | — | *pending measurement* |
23+
24+
⏳ **Pending measurement** (a manifest exists; not yet run on the rig): Busbar (main). These land here as their runs complete — nothing is hidden.
2225

2326
Two throughput numbers: **max proxy RPS** (instant upstream — raw forwarding speed) and **sustained RPS @20ms** (AIGatewayBench's metric — concurrent in-flight capacity under realistic LLM latency).
24-
**✕** = did not serve under load (0 successful req/s). &nbsp; **0** = came up, but no tested concurrency held p99 < 1 s with <0.1% errors.
27+
**✕** = did not serve under load (0 successful req/s). &nbsp; **0** = came up, but no tested concurrency held p99 < 1 s with <0.1% errors. &nbsp; **⏳** = a manifest exists but it hasn't been run on the rig yet.
2528

26-
![added_latency](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/added_latency.png?v=202607220450)
29+
![added_latency](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/added_latency.png?v=202607220538)
2730

28-
![rps_max_proxy](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/rps_max_proxy.png?v=202607220450)
31+
![rps_max_proxy](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/rps_max_proxy.png?v=202607220538)
2932

30-
![rps_sustained_20ms](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/rps_sustained_20ms.png?v=202607220450)
33+
![rps_sustained_20ms](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/rps_sustained_20ms.png?v=202607220538)
3134

32-
![memory_rss](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/memory_rss.png?v=202607220450)
35+
![memory_rss](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/memory_rss.png?v=202607220538)
3336

34-
![rps_per_dollar](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/rps_per_dollar.png?v=202607220450)
37+
![rps_per_dollar](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/rps_per_dollar.png?v=202607220538)
3538

36-
![cost_per_million](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/cost_per_million.png?v=202607220450)
39+
![cost_per_million](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/cost_per_million.png?v=202607220538)
3740

3841
---
3942
Method: added latency = gateway p99 − direct-to-mock p99 at concurrency 1; RPS ceiling = highest sustained req/s with p99 < 1 s and <0.1% errors; RSS idle = after first 200, peak = under sustained load. Same box, same mock, same load, one gateway at a time. Source refs pinned in `gateways/versions.env`; the built commit is in each row.
4043

41-
<sub>Page + charts regenerated **2026-07-22 04:50 UTC** from the raw `results/*.json`.</sub>
44+
<sub>Page + charts regenerated **2026-07-22 05:38 UTC** from the raw `results/*.json`.</sub>

‎results/reports/top5/README.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,19 +13,19 @@ Every number below is regenerated from the raw `results/*.json` — re-run `run-
1313
| [APISIX](https://github.com/apache/apisix) | 486 µs | 17,326 | 19,117 | 181 MiB | 754 MiB | `apache/apisix:3.17.0-debian (@sha256:6cbf65f30` |
1414

1515
Two throughput numbers: **max proxy RPS** (instant upstream — raw forwarding speed) and **sustained RPS @20ms** (AIGatewayBench's metric — concurrent in-flight capacity under realistic LLM latency).
16-
![added_latency](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_added_latency.png?v=202607220450)
16+
![added_latency](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_added_latency.png?v=202607220538)
1717

18-
![rps_max_proxy](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_rps_max_proxy.png?v=202607220450)
18+
![rps_max_proxy](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_rps_max_proxy.png?v=202607220538)
1919

20-
![rps_sustained_20ms](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_rps_sustained_20ms.png?v=202607220450)
20+
![rps_sustained_20ms](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_rps_sustained_20ms.png?v=202607220538)
2121

22-
![memory_rss](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_memory_rss.png?v=202607220450)
22+
![memory_rss](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_memory_rss.png?v=202607220538)
2323

24-
![rps_per_dollar](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_rps_per_dollar.png?v=202607220450)
24+
![rps_per_dollar](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_rps_per_dollar.png?v=202607220538)
2525

26-
![cost_per_million](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_cost_per_million.png?v=202607220450)
26+
![cost_per_million](https://raw.githubusercontent.com/GetBusbar/benchmarking/main/results/top5_cost_per_million.png?v=202607220538)
2727

2828
---
2929
Method: added latency = gateway p99 − direct-to-mock p99 at concurrency 1; RPS ceiling = highest sustained req/s with p99 < 1 s and <0.1% errors; RSS idle = after first 200, peak = under sustained load. Same box, same mock, same load, one gateway at a time. Source refs pinned in `gateways/versions.env`; the built commit is in each row.
3030

31-
<sub>Page + charts regenerated **2026-07-22 04:50 UTC** from the raw `results/*.json`.</sub>
31+
<sub>Page + charts regenerated **2026-07-22 05:38 UTC** from the raw `results/*.json`.</sub>

0 commit comments

Comments
 (0)