You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 373c8cc
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: results/reports/all/README.md
+24-18Lines changed: 24 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,20 +8,20 @@ Every number below is regenerated from the raw `results/*.json` - re-run `run-al
8
8
|---|--:|--:|--:|--:|---|
9
9
|[LiteLLM · Rust](https://github.com/BerriAI/litellm)| 105 µs | 44,382 <sub>(+1% from 1 ms to no bound)</sub> | - | - |`litellm-ai-gateway`|
10
10
|[agentgateway](https://github.com/agentgateway/agentgateway)| 229 µs | 25,041 <sub>(+6% from 1 ms to no bound)</sub> | 25 MiB | 47 MiB |`ghcr.io/agentgateway/agentgateway:v1.4.0`|
11
+
|[AISIX (api7)](https://github.com/api7/aisix)| 279 µs | 17,428 <sub>(+4% from 1 ms to no bound)</sub> | 67 MiB | 350 MiB |`target/release/aisix`|
11
12
|[Helicone](https://github.com/Helicone/ai-gateway)| 322 µs | 14,826 <sub>(+6% from 1 ms to no bound)</sub> | 43 MiB | 56 MiB |`target/release/ai-gateway`|
12
13
|[Kong](https://github.com/Kong/kong)| 389 µs | 21,867 <sub>(+63% from 1 ms to no bound)</sub> | 382 MiB | 595 MiB |`kong:3.9.3`|
13
14
|[APISIX](https://github.com/apache/apisix)| 446 µs | 20,229 <sub>(+74% from 1 ms to no bound)</sub> | 180 MiB | 211 MiB |`apache/apisix:3.17.0-debian`|
14
15
|[Bifrost](https://github.com/maximhq/bifrost)| 900 µs | 5,198 <sub>(+225% from 1 ms to no bound)</sub> | 235 MiB | 815 MiB |`maximhq/bifrost:v1.6.6`|
16
+
|[GoModel](https://github.com/ENTERPILOT/GOModel)| 2,020 µs | 1,828 <sub>(+52% from 5 ms to no bound)</sub> | 51 MiB | 86 MiB |`enterpilot/gomodel:0.1.63`|
⏳ **Pending measurement** (a manifest exists; not yet run on the rig): AISIX (api7), Busbar, GoModel, LiteLLM · Python, Portkey. These land here as their runs complete - nothing is hidden.
24
+
⏳ **Pending measurement** (a manifest exists; not yet run on the rig): Busbar, LiteLLM · Python, Portkey. These land here as their runs complete - nothing is hidden.
25
25
26
26
**Throughput is a curve, not a number.** The column above is one reading of each gateway's concurrency sweep: the most req/s it carried while 99% of requests finished under **10 ms** and it failed **none** it accepted. The same sweep is published at 5 tail-latency bounds (1 ms, 5 ms, 10 ms, 50 ms, 100 ms) plus with no bound at all, and the shape across them is the comparison that matters: a gateway already at its ceiling at 1 ms is a different machine from one that doubles when given 5 ms. See the frontier-shape chart. **≥** on a number means the sweep's top rung won, so that rate is a floor and no ceiling was established.
27
27
**✕** = did not serve under load (0 successful req/s). **0** = came up, but no tested concurrency held p99 < 10 ms while failing none of the requests it accepted. **⏳** = a manifest exists but it hasn't been run on the rig yet.
@@ -34,10 +34,12 @@ The most req/s each gateway carried while 99% of requests finished under the col
**✕** cells are measured refusals, not gaps: the gateway was offered the load and could not do the thing (buffered instead of streaming, rejected the Anthropic shape, or has no native key/limit governance). **n/a** = that suite hasn't been run for this gateway yet.
Method: added latency = gateway p99 − direct-to-mock p99 at concurrency 1; a frontier reading = the highest req/s any probed concurrency carried while 99% of requests finished under the STATED bound and the gateway failed none it accepted (readings are published at 1, 5, 10, 50, 100 ms and with no bound; the columns above use 10 ms, and every caption names the bound it used); cost figures divide that 10 ms reading by $0.1632/hr for the pinned 4-core (m7g.xlarge) slice; RSS idle = after first 200, steady state = the level the RSS settled at under load. Same box, same mock, same load, one gateway at a time. Each gateway's source ref is pinned in its own `gateways/<name>/definition.json`; the built commit is in each row.
113
119
114
-
<sub>Page + charts regenerated **2026-07-30 06:58 UTC** from the raw `results/*.json`.</sub>
120
+
<sub>Page + charts regenerated **2026-07-30 07:29 UTC** from the raw `results/*.json`.</sub>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4&size=40){kind=avatar}
0 commit comments