Skip to content

Commit b12be2a

Browse files
MattJacksonMattJackson
committed
bench: add busbar 1.5.0 as its own entrant
getbusbar/busbar:1.5.0 was published today (Docker Hub, 05:29 UTC), so it is the same KIND of artifact as every other row - a released image, not a source build. Its own gateway key, not a re-run of 'busbar'. Snapshots are keyed by gateway, so measuring 1.5.0 under the existing key would overwrite the 1.4.1 row rather than sit beside it, and the comparison is the entire point. The config drops max_concurrent, and that is the minimal config for this version rather than a tuning choice. In v1.4.1 the field is a bare usize with no serde default, so that manifest MUST set it. In v1.5.0 it is Option<usize> and the source says outright: 'max_concurrent is an OPT-IN limiter: omitted (None) = UNBOUNDED'. Carrying 8000 over would not have been neutral - busbar's stream sweep probes past c=8192, so the cap binds inside the measured range and would have handicapped 1.5.0 against what a user gets by installing it. The config-necessity rule decides this on its own: every gateway config is the bare minimum required to run.
1 parent f2fda44 commit b12be2a

5 files changed

Lines changed: 172 additions & 0 deletions

File tree

‎gateways/busbar-150/config.gen.yaml.tmpl‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
listen: "127.0.0.1:{GW_PORT}"
2+
providers:
3+
mock-openai:
4+
api_key_env: BENCH_MOCK_KEY
5+
mock-responses:
6+
api_key_env: BENCH_MOCK_KEY
7+
mock-anthropic:
8+
api_key_env: BENCH_MOCK_KEY
9+
mock-gemini:
10+
api_key_env: BENCH_MOCK_KEY
11+
mock-cohere:
12+
api_key_env: BENCH_MOCK_KEY
13+
mock-bedrock:
14+
api_key_env: BENCH_BEDROCK_KEY
15+
models:
16+
gpt-4o-mini:
17+
# OOTB canonical (OpenAI) lane — the perf/memory/throughput default.
18+
#
19+
# NO max_concurrent, AND THAT IS THE MINIMAL CONFIG FOR THIS VERSION. In v1.4.1 the field was
20+
# REQUIRED (a bare usize, no serde default) so the 1.4.1 manifest must set it, and 8000 sat above
21+
# the sweep's winning band as a ceiling rather than a limiter. In v1.5.0 it is
22+
# `Option<usize>` and the source says so outright: "max_concurrent is an OPT-IN limiter: omitted
23+
# (None) = UNBOUNDED". So the bare minimum required to run no longer includes it.
24+
#
25+
# Carrying 8000 over would not have been neutral: busbar's stream sweep probes past c=8192, so
26+
# the cap binds inside the measured range and would have handicapped 1.5.0 against the very
27+
# thing a user gets by installing it. The config-necessity rule - every gateway config is the
28+
# bare minimum required to run - decides this on its own.
29+
# max_requests: -1 = unmetered.
30+
provider: mock-openai
31+
gpt-4o-mini-responses:
32+
provider: mock-responses
33+
gpt-4o-mini-anthropic:
34+
provider: mock-anthropic
35+
gpt-4o-mini-gemini:
36+
provider: mock-gemini
37+
gpt-4o-mini-cohere:
38+
provider: mock-cohere
39+
gpt-4o-mini-bedrock:
40+
provider: mock-bedrock

‎gateways/busbar-150/definition.json‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
{
2+
"auth": "bench-token",
3+
"class": "Control plane",
4+
"config": [],
5+
"config_files": [
6+
{
7+
"output": "providers.gen.yaml",
8+
"template": "providers.gen.yaml.tmpl"
9+
},
10+
{
11+
"output": "config.gen.yaml",
12+
"template": "config.gen.yaml.tmpl"
13+
}
14+
],
15+
"display": "Busbar 1.5.0",
16+
"egress": [
17+
"openai",
18+
"openai-responses",
19+
"anthropic",
20+
"gemini",
21+
"cohere",
22+
"bedrock"
23+
],
24+
"egress_models": {
25+
"anthropic": "gpt-4o-mini-anthropic",
26+
"bedrock": "gpt-4o-mini-bedrock",
27+
"cohere": "gpt-4o-mini-cohere",
28+
"gemini": "gpt-4o-mini-gemini",
29+
"openai": "gpt-4o-mini",
30+
"openai-responses": "gpt-4o-mini-responses"
31+
},
32+
"lang": "Rust",
33+
"launch": {
34+
"args": [],
35+
"image": "getbusbar/busbar:1.4.1",
36+
"kind": "docker",
37+
"mounts": [
38+
{
39+
"container_path": "/etc/busbar/config.yaml",
40+
"host_path": "config.gen.yaml",
41+
"read_only": true
42+
},
43+
{
44+
"container_path": "/etc/busbar/providers.yaml",
45+
"host_path": "providers.gen.yaml",
46+
"read_only": true
47+
}
48+
]
49+
},
50+
"matrix": [
51+
"111111",
52+
"111111",
53+
"111111",
54+
"111111",
55+
"111111",
56+
"111111"
57+
],
58+
"matrix_note": "busbar declares full 6x6 translation support",
59+
"model": "gpt-4o-mini",
60+
"name": "busbar",
61+
"path": "/v1/chat/completions",
62+
"port": 8080,
63+
"repo": "https://github.com/GetBusbar/busbar",
64+
"runtime": {
65+
"container": "busbar-bench",
66+
"image": "getbusbar/busbar:1.5.0",
67+
"kind": "docker"
68+
}
69+
}

‎gateways/busbar-150/env‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
BENCH_MOCK_KEY=dummy
2+
BENCH_BEDROCK_KEY=AKIAMOCKACCESSKEY:mock-secret-access-key

‎gateways/busbar-150/providers.gen.yaml.tmpl‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
mock-openai:
2+
protocol: openai
3+
base_url: http://127.0.0.1:{MOCK_PORT}
4+
api_key_env: BENCH_MOCK_KEY
5+
error_map: {{}}
6+
mock-responses:
7+
protocol: responses
8+
base_url: http://127.0.0.1:{MOCK_PORT}
9+
api_key_env: BENCH_MOCK_KEY
10+
error_map: {{}}
11+
mock-anthropic:
12+
protocol: anthropic
13+
base_url: http://127.0.0.1:{MOCK_PORT}
14+
api_key_env: BENCH_MOCK_KEY
15+
error_map: {{}}
16+
mock-gemini:
17+
protocol: gemini
18+
base_url: http://127.0.0.1:{MOCK_PORT}
19+
api_key_env: BENCH_MOCK_KEY
20+
error_map: {{}}
21+
mock-cohere:
22+
protocol: cohere
23+
base_url: http://127.0.0.1:{MOCK_PORT}
24+
api_key_env: BENCH_MOCK_KEY
25+
error_map: {{}}
26+
mock-bedrock:
27+
protocol: bedrock
28+
base_url: http://127.0.0.1:{MOCK_PORT}
29+
api_key_env: BENCH_BEDROCK_KEY
30+
error_map: {{}}

‎gateways/busbar-150/why‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Why every setting, env var, header and command this gateway uses is here.
2+
#
3+
# The rule, from the published methodology: a setting may appear only if it is needed to boot the
4+
# process at all, to point an upstream at the test mock instead of a real provider, to expose an
5+
# ingress path the matrix exercises, or to bind the port and cores the rig requires. One claim per
6+
# line: <pattern> <boot|upstream|ingress|bind>. Anything unclaimed is a lint failure, and a claim
7+
# that matches nothing is a stale claim and also a failure.
8+
#
9+
# Migrated from GW_CONFIG_WHY in gateway.sh, which is dead code the lint was still reading.
10+
listen bind # the port the harness drives
11+
providers upstream # the provider table
12+
mock-* upstream # one provider per protocol, each base_url -> the mock
13+
protocol upstream # which upstream dialect that provider speaks
14+
base_url upstream # -> the mock
15+
api_key_env boot # names the env var holding the credential; a provider without one fails load
16+
error_map boot # REQUIRED per provider in v1.4.1 and still in v1.5.0 - no serde default
17+
models ingress # the client-facing model table
18+
gpt-4o-mini* ingress # the model name each matrix column asks for; selects that column's provider
19+
provider upstream # which provider backs this model
20+
# max_concurrent REMOVED for v1.5.0. It was a bare `usize` with no serde default in v1.4.1, so that
21+
# manifest must set it; in v1.5.0 it is `Option<usize>` and the source states
22+
# "omitted (None) = UNBOUNDED", so the bare minimum to run no longer includes it.
23+
# Keeping the 8000 from the 1.4.1 config would not have been neutral either: the
24+
# stream sweep probes past c=8192, so the cap binds inside the measured range.
25+
BENCH_MOCK_KEY boot # the credential api_key_env names (dummy; the mock ignores it)
26+
BENCH_BEDROCK_KEY boot # bedrock SigV4 ACCESS:SECRET pair, same
27+
governance boot
28+
enabled boot # without it admin_token is inert and the key mint 401s
29+
db_path boot # the default file cannot be opened on the bench's read-only working dir
30+
admin_token boot # activates the admin plane the lane mints its virtual key over
31+
admin_listen bind # the admin plane has its own listener; the mint has to know the port

0 commit comments

Comments
 (0)